Skip to main content
SpringerLink
Log in

Image Deblurring with Coupled Dictionary Learning

  • Published:
International Journal of Computer Vision Aims and scope Submit manuscript

Abstract

Image deblurring is a challenging problem in vision computing. Traditionally, this task is addressed as an inverse problem that is enclosed into the image itself. This paper presents a learning-based framework where the knowledge hidden in huge amounts of available data is explored and exploited for image deblurring. To this end, our algorithm is developed under the conceptual framework of coupled dictionary learning. Specifically, given pairs of blurred image patches and their corresponding clear ones, a learning model is constructed to learn a pair of dictionaries. Among them, one dictionary is responsible for the representation of clear images, while the other is responsible for that of the blurred images. Theoretically, the learning model is analyzed with coupled sparse representations for training samples. As the atoms of these dictionaries are coupled together one-by-one, the reconstruction information can be transmitted between the clear and blurry images. In application phase, the blurry dictionary is employed to reconstruct linearly the blurry image to be restored. Then, the reconstruction coefficients are kept unchanged along with the clear dictionary to restore the final results. The main advantage of our approach lies in that it works in the case of unknown blur kernels. Comparative experiments indicate the validity of our approach.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18

Similar content being viewed by others

Notes

  1. KSVD-DR refines the high-resolution dictionary by using the overlapped patches. Here we can not do this treatment since the patches are sampled from different images.

  2. Available at: http://msm.cais.ntu.edu.sg/LSCBIR.

  3. The executable file is downloaded from the webpage: http://www.cse.cuhk.edu.hk/~leojia/programs/deblurring/deblurring.htm.

References

  • Afonso, M. V., Bioucas-Dias, J. M., & Figueiredo, M. (2010). Fast image recovery using variable splitting and constrained optimization. IEEE Transactions on Image Processing, 19(9), 2345–2356.

    Article  MathSciNet  Google Scholar 

  • Aharon, M., Elad, M., & Bruckstein, A. M. (2006). K-svd: An algorithm for designing of over-complete dictionaries for sparse representationn. IEEE Transactions on Signal Processing, 54(11), 4311–4322.

    Article  Google Scholar 

  • Bar, L., Sochen, N., & Kiryati, N. (2007). Restoration of images with piecewise space-variant blur. In International conference on scale space and variational methods in computer vision (pp. 533–544). Ischia, Italy.

  • Barbero, A., & Sra, S. (2011). Fast newton-type methods for total variation regularization. In International conference on machine learning (pp. 313–320). Bellevue, Washington, USA.

  • Batra, D., Kowdle, A., Parikh, D., Luo, J., & Chen, T. (2010). icoseg: Interactive co-segmentation with intelligent scribble guidance. In IEEE conference on computer vision and pattern recognition (pp. 3169–3176). San Francisco, CA, USA.

  • Batra, D., Kowdle, A., Parikh, D., Luo, J., & Chen, T. (2011). Interactively co-segmentating topically related images with intelligent scribble guidance. International Journal of Computer Vision, 93(3), 273–292.

    Article  Google Scholar 

  • Beck, A., & Teboulle, M. (2009a). Fast gradient-based algorithms for constrained total variation image denoising and deblurring problems. IEEE Transactions on Image Processing, 18(11), 2419–2434.

    Article  MathSciNet  Google Scholar 

  • Beck, A., & Teboulle, M. (2009b). A fast iterative shrinkage-thresholding algorithm for linear inverse problems. SIAM Journal on Imaging Sciences, 2(1), 183–202.

    Article  MathSciNet  MATH  Google Scholar 

  • Bioucas-Dias, J. M., & Figueiredo, M. (2007). A new twist: Two-step iterative shrinkage/thresholding algorithms for image restoration. IEEE Transactions on Image Processing, 16(12), 2992–3004.

    Article  MathSciNet  Google Scholar 

  • Bronstein, M. M., Bronstein, A. M., Zibulevsky, M., & Zeevi, Y. Y. (2005). Blind deconvolution of images using optimal sparse representations. IEEE Transactions on Image Processing, 14(6), 726–736.

    Article  Google Scholar 

  • Chantas, G. K., Galatsanos, N. P., Molina, R., & Katsaggelos, A. K. (2010). Variational bayesian image restoration with a product of spatially weighted total variation image priors. IEEE Transactions on Image Processing, 19(2), 351–362.

    Article  MathSciNet  Google Scholar 

  • Couzinie-Devy, F., Mairal, J., Bach, F., & Ponce, J. (2011). Dictionary learning for deblurring and digital zoom. CoRR. arXiv:1110.0957.

  • Dabov, K., Foi, A., Katkovnik, V., & Egiazarian, K. (2007). Image denoising by sparse 3d transform-domain collaborative filtering. IEEE Transactions on Image Processing, 16(8), 2080–2095.

    Article  MathSciNet  Google Scholar 

  • Danielyan, A., Katkovnik, V., & Egiazarian, K. (2012). Bm3d frames and variational image deblurring. IEEE Transactions on Image Processing, 21(4), 1715–1728.

    Article  MathSciNet  Google Scholar 

  • Dong, W., Zhang, L., Shi, G., & Li, X. (2013). Nonlocally centralized sparse representation for image restoration. IEEE Transactions on Image Processing, 22(4), 1620–1630.

    Article  MathSciNet  Google Scholar 

  • Dong, W., Zhang, L., Shi, G., & Wu, X. (2011). Image deblurring and super-resolution by adaptive sparse domain selection and adaptive regularization. IEEE Transactions on Image Processing, 20(7), 1838–1857.

    Article  MathSciNet  Google Scholar 

  • Elad, M., & Aharon, M. (2006). Image denoising via sparse and redundant representations over learned dictionaries. IEEE Transactions on Image Processing, 15(12), 3736–3745.

    Article  MathSciNet  Google Scholar 

  • Elad, M., Figueiredo, M., & Ma, Y. (2010). On the role of sparse and redundant representations in image processing. Proceedings of the IEEE, 98(6), 972–982.

    Article  Google Scholar 

  • Engan, K., Aase, S. O., & Husoy, J. H. (2000). Multi-frame compression: Theory and design. EURASIP Signal Processing, 80(10), 2121–2140.

    Article  MATH  Google Scholar 

  • Fergus, R., Singh, B., Hertzmann, A., Roweis, S. T., & Freeman, W. T. (2006). Removing camera shake from a single photograph. ACM Transactions on Graphics (SIGGRAPH), 25(3), 787–794.

    Article  Google Scholar 

  • Goldluecke, B., & Cremers, D. (2010). An approach to vectorial total variation based on geometric measure theory. In IEEE international conference on computer vision and pattern recognition (pp. 327–333). San Francisco, CA, USA.

  • Gorodnitsky, I. F., & Rao, B. D. (1997). Sparse signal reconstruction from limited data using focuss: A re-weighted norm minimization algorithm. IEEE Transactions on Signal Processing, 45(3), 600–616.

    Article  Google Scholar 

  • He, L., Qi, H., & Zaretzki, R. (2013). Beta process joint dictionary learning for coupled feature spaces with application to single image super-resolution. In IEEE conference on computer vision and pattern recognition (pp. 345–352). Portland, Oregon, USA.

  • Hu, Z., Huang, J. B., & Yang, M. H. (2010). Single image deblurring with adaptive dictionary learning. In IEEE international conference on image processing (pp. 1169–1172). Hong Kong, China.

  • Jenatton, R., Mairal, J., Obozinski, G., & Bach, F. (2011). Proximal methods for hierarchical sparse coding. Journal of Machine Learning Research, 12, 2297–2334.

    MathSciNet  MATH  Google Scholar 

  • Jia, K., Wang, X., & Tang, X. (2013). Image transformation based on learning dictionaries across image spaces. IEEE Transactions on Pattern Analysis and Machine Intelligence, 35(2), 367–380.

    Article  Google Scholar 

  • Joshi, N., Kang, S. B., Zitnick, C. L., & Szeliski, R. (2010). Image deblurring using inertial measurement sensors. In SIGGRAPH (pp. 1–8). Los Angeles, CA, USA.

  • Joshi, N., Zitnick, C. L., Szeliski, R., & Kriegman, D. J. (2009). Image deblurring and denoising using color priors. In International conference on computer vision and pattern recognition (pp. 1–8). Miami, FL, USA.

  • Kamilov, U., Bostan, E., & Unser, M. (2012). Wavelet shrinkage with consistent cycle spinning generalizes total variation denoising. IEEE Signal Processing Letters, 19(4), 187–190.

    Article  Google Scholar 

  • Kenig, T., Kam, Z., & Feuer, A. (2010). Blind image deconvolution using machine learning for three-dimensional microscopy. IEEE Transactions on Pattern Analysis and Machine Intelligence, 32(12), 2191–2204.

    Article  Google Scholar 

  • Levin, A., Fergus, R., Durand, F., & Freeman, W. T. (2007). Coded exposure photography: Motion deblurring using fluttered shutter. ACM Transactions on Graphics (SIGGRAPH), 26(3), 70:1–70:9.

    Article  Google Scholar 

  • Levin, A., Weiss, Y., Durand, F., & Freeman, W. T. (2009). Understanding and evaluating blind deconvolution algorithms. In International conference on computer vision and pattern recognition (pp. 1–8). Miami, FL, USA.

  • Li, D., Mersereau, R. M., & Simske, S. (2007). Blind image deconvolution through support vector regression. IEEE Transactions on Neural Networks, 18(3), 931–935.

    Article  Google Scholar 

  • Lou, Y., Bertozzi, A. L., & Soatto, S. (2011). Direct sparse deblurring. Journal of Mathematical Imaging and Vision, 39(1), 1–12.

    Article  MathSciNet  MATH  Google Scholar 

  • Lucy, L. (1974). An iterative technique for the rectification of observed distributions. The Astronomical Journal, 79(6), 745–754.

    Article  Google Scholar 

  • Ma, L., Moisan, L., Yu, J., & Zeng, T. (2013). A dictionary learning approach for poisson image deblurring. IEEE Transactions on Image Processing, 32(7), 1277–1289.

    Article  Google Scholar 

  • Ma, S., Yin, W., Zhang, Y., & Chakraborty, A. (2008). An efficient algorithm for compressed MR imaging using total variation and wavelets. In IEEE international conference on computer vision and pattern recognition (pp. 1–8). Anchorage, Alaska, USA.

  • Mairal, J., Bach, F., & Ponce, J. (2012). Task-driven dictionary learning. IEEE Transactions on Pattern Analysis and Machine Intelligence, 34(4), 791–804.

    Article  Google Scholar 

  • Mairal, J., Bach, F., Ponce, J., & Sapiro, G. (2010). Online learning for matrix factorization and sparse coding. Journal of Machine Learning Research, 11, 19–60.

    MathSciNet  MATH  Google Scholar 

  • Mairal, J., Bach, F., Ponce, J., Sapiro, G., & Zisserman, A. (2009). Non-local sparse models for image restoration. In IEEE international conference on computer vision (pp. 2272–2279). Kyoto, Japan.

  • Mairal, J., Elad, M., & Sapiro, G. (2008). Sparse representation for color image restoration. IEEE Transactions on Image Processing, 17(1), 53–69.

    Article  MathSciNet  Google Scholar 

  • Mairal, J., Jenatton, R., Obozinski, G., & Bach, F. (2011). Convex and network flow optimization for structured sparsity. Journal of Machine Learning Research, 12, 2681–2720.

    MathSciNet  MATH  Google Scholar 

  • Peleg, T., Eldar, Y., & Elad, M. (2013). Exploiting statistical dependencies in sparse representations for signal recovery. IEEE Transactions on Signal Processing, 60(5), 2286–2303.

    Article  MathSciNet  Google Scholar 

  • Portilla, J. (2009). Image restoration through l0 analysis-based sparse optimization in tight frames. In IEEE international conference on image processing (pp. 3909–3912). Cairo, Egypt.

  • Richardson, W. H. (1972). Bayesian-based iterative method of image restoration. Journal of the Optical Society of America, 62(1), 55–59.

    Article  Google Scholar 

  • Rodriguez, P., & Wohlberg, B. (2009). Efficient minimization method for a generalized total variation functional. IEEE Transactions on Image Processing, 18(2), 322–332.

    Article  MathSciNet  Google Scholar 

  • Rubinstein, R., Peleg, T., & Elad, M. (2013). Analysis k-svd: A dictionary-learning algorithm for the analysis sparse model. IEEE Transactions on Signal Processing, 61(3), 661–677.

    Article  MathSciNet  Google Scholar 

  • Rubinstein, R., Zibulevsky, M., & Elad, M. (2010). Double sparsity: Learning sparse dictionaries for sparse signal approximation. IEEE Transactions on Signal Processing, 58(3), 1553–1564.

    Article  MathSciNet  Google Scholar 

  • Rudin, L. I., Osher, S., & Fatemi, E. (1992). Nonlinear total variation based noise removal algorithms. Physica D, 60(1), 259–268.

    Article  MATH  Google Scholar 

  • Shan, Q., Jia, J., & Agarwala, A. (2008). High-quality motion deblurring from a single image. ACM Transactions on Graphics (SIGGRAPH), 27(3), 73:1–73:10.

    Article  Google Scholar 

  • Su, M., & Basu, M. (2002). A hybrid learning system for image deblurring. Pattern Recognition, 35(12), 2881–2894.

    Article  MATH  Google Scholar 

  • Takeda, H., Farsiu, S., & Milanfar, P. (2008). Deblurring using regularized locally adaptive kernel regression. IEEE Transactions on Image Processing, 17(4), 550–563.

    Article  MathSciNet  Google Scholar 

  • Tropp, J. A. (2004). Greed is good: Algorithmic results for sparse approximation. IEEE Transactions on Information Theory, 50(10), 2231–2242.

    Article  MathSciNet  MATH  Google Scholar 

  • Wang, S., Zhang, L., Liang, Y., & Pan, Q. (2012). Semi-coupled dictionary learning with applications to image super-resolution and photo-sketch synthesis. In IEEE conference on computer vision and pattern recognition (pp. 2216–2223). Providence, Rhode Island, USA.

  • Wang, Z., & Li, Q. (2011). Information content weighting for perceptual image quality assessment. IEEE Transactions on Image Processing, 20(5), 1185–1198.

    Article  MathSciNet  Google Scholar 

  • Wen, Y. W., Chan, R. H., & Yip, A. M. (2012). A primal-dual method for total-variation-based wavelet domain inpainting. IEEE Transactions on Image Processing, 21(1), 106–114.

    Article  MathSciNet  Google Scholar 

  • Whyte, O., Sivic, J., Zisserman, A., & Ponce, J. (2012). Non-uniform deblurring for shaken images. International Journal of Computer Vision, 98(2), 168–186.

    Article  MathSciNet  MATH  Google Scholar 

  • Xiang, S., Nie, F., & Zhang, C. (2010). Semi-supervised classification via local spline regression. IEEE Transactions on Pattern Analysis and Machine Intelligence, 32(11), 2039–2053.

  • Xu, L., Zheng, S., & Jia, J. (2013). Unnatural l0 sparse representation for natural image deblurring. In IEEE conference on computer vision and pattern recognition (pp. 1107–1114). Portland, Oregon, USA.

  • Yang, J., Wang, Z., Lin, Z., Cohen, S., & Huang, T. S. (2012). Coupled dictionary training for image super-resolution. IEEE Transactions on Image Processing, 21(8), 3467–3478.

  • Yang, J., Wright, J., Huang, T. S., & Ma, Y. (2010). Image super-resolution via sparse representation. IEEE Transactions on Signal Processing, 19(11), 2861–2873.

  • Yuan, L., Sun, J., Quan, L., & Shum, H. Y. (2007). Image deblurring with blurred/noisy image pairs. ACM Transactions on Graphics (SIGGRAPH), 26(3), 1–10.

    Article  MATH  Google Scholar 

  • Zeyde, R., Elad, M., & Protter, M. (2010). On single image scale-up using sparse-representations. In Proceedings of the 7th international conference on curves and surfaces (pp. 711–730). Avignon, France.

  • Zhang, H., Wipf, D. P., & Zhang, Y. (2013). Multi-image blind deblurring using a coupled adaptive sparse prior. In IEEE conference on computer vision and pattern recognition (pp. 1051–1058). Portland, Oregon, USA.

  • Zhang, H., Yang, J., Zhang, Y., & Huang, T. S. (2011). Sparse representation based blind image deblurring. In IEEE international conference on multimedia and expo (pp. 1–6). Barcelona, Spain.

Download references

Acknowledgments

This work was supported by the National Natural Science Foundation of China (Grant Nos. 61272331, 91338202, 61370039, and 91120301).

Author information

Authors and Affiliations

  1. National Laboratory of Pattern Recognition, Institute of Automation, Chinese Academy of Sciences, Beijing, 100190, China

    Shiming Xiang, Gaofeng Meng, Ying Wang & Chunhong Pan

  2. Tsinghua National Laboratory for Information Science and Technology (TNList), Department of Automation, Tsinghua University, Beijing, 100084, China

    Changshui Zhang

Authors
  1. Shiming Xiang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Gaofeng Meng
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ying Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Chunhong Pan
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Changshui Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Shiming Xiang.

Additional information

Communicated by Julien Mairal, Francis Bach and Michael Elad.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Xiang, S., Meng, G., Wang, Y. et al. Image Deblurring with Coupled Dictionary Learning. Int J Comput Vis 114, 248–271 (2015). https://doi.org/10.1007/s11263-014-0755-z

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11263-014-0755-z

Keywords

Search

Navigation