Skip to main content
Springer Nature Link
Log in

Total variation image deblurring with space-varying kernel

  • Published:
Computational Optimization and Applications Aims and scope Submit manuscript

Abstract

Image deblurring techniques based on convex optimization formulations, such as total-variation deblurring, often use specialized first-order methods for large-scale nondifferentiable optimization. A key property exploited in these methods is spatial invariance of the blurring operator, which makes it possible to use the fast Fourier transform (FFT) when solving linear equations involving the operator. In this paper we extend this approach to two popular models for space-varying blurring operators, the Nagy–O’Leary model and the efficient filter flow model. We show how splitting methods derived from the Douglas–Rachford algorithm can be implemented with a low complexity per iteration, dominated by a small number of FFTs.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

References

  1. Almeida, M.S.C., Figueiredo, M.A.T.: Deconvolving images with unknown boundaries using the alternating direction method of multipliers. IEEE Trans. Image Process. 22(8), 3074–3086 (2013)

    Article  MathSciNet  Google Scholar 

  2. Bardsley, J., Jefferies, S., Nagy, J., Plemmons, R.: A computational method for the restoration of images with an unknown, spatially-varying blur. Opt. Express 14(5), 1767–1782 (2006)

    Article  Google Scholar 

  3. Hadj, S.B., Féraud, L.B.: Restoration method for spatially variant blurred images. Rapport de recherche RR-7654, INRIA (2011)

  4. Hadj, S.B., Blanc-Féraud, L.: Modeling and removing depth variant blur in 3D fluorescence microscopy. In: 2012 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 689–692. IEEE (2012)

  5. Hadj, S.B., Blanc-Féraud, L., Aubert, G.: Space variant blind image restoration. SIAM J. Imaging Sci. 7(4), 2196–2225 (2014)

  6. Boyd, S., Parikh, N., Chu, E., Peleato, B., Eckstein, J.: Distributed optimization and statistical learning via the alternating direction method of multipliers. Found. Trends Mach. Learn. 3(1), 1–122 (2011)

    Article  MATH  Google Scholar 

  7. Briceno-Arias, L.M., Combettes, P.L.: A monotone \(+\) skew splitting model for composite monotone inclusions in duality. SIAM J. Optim. 21(4), 1230–1250 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  8. Chakrabarti, A., Zickler, T., Freeman, W. T.: Analyzing spatially-varying blur. In: 2010 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2512–2519. IEEE (2010)

  9. Chambolle, A., Pock, T.: A first-order primal-dual algorithm for convex problems with applications to imaging. J. Math. Imaging Vis. 40, 120–145 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  10. Chan, R.H., Tao, M., Yuan, X.: Constrained total variation deblurring methods and fast algorithms based on alternating direction method of multipliers. SIAM J. Imaging Sci. 6(1), 680–697 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  11. Cho, S., Lee, S.: Fast motion deblurring. ACM Trans. Graph. 28(5), 145:1–145:8 (2009)

    Article  Google Scholar 

  12. Combettes, P.L., Pesquet, J.-C.: A Douglas–Rachford splitting approach to nonsmooth convex variational signal recovery. IEEE J. Sel. Top. Signal Process. 1(4), 564–574 (2007)

    Article  Google Scholar 

  13. Combettes, P.L., Wajs, V.R.: Signal recovery by proximal forward–backward splitting. Multiscale Model. Simul. 4(4), 1168–1200 (2005)

    Article  MathSciNet  MATH  Google Scholar 

  14. Condat, L.: A primal-dual splitting method for convex optimization involving Lipschitzian, proximable and linear composite terms. J. Optim. Theory Appl. 158(2), 460–479 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  15. Denis, L., Thiebaut, E., Soulez, F.: Fast model of space-variant blurring and its application to deconvolution in astronomy. In: 2011 18th IEEE International Conference on Image Processing (ICIP), pp. 2817–2820 (2011)

  16. Ekeland, I., Témam, R.: Convex Analysis and Variational Problems, Vol. 28 of Classics in Applied Mathematics. Society for Industrial and Applied Mathematics, Philadelphia (1999). (First published in 1976 by North-Holland)

    Book  Google Scholar 

  17. Escande, P., Weiss, P., Malgouyres, F.: Image restoration using sparse approximations of spatially varying blur operators in the wavelet domain. J. Phys. Conf. Ser. 464, 012004 (2013)

    Article  Google Scholar 

  18. Fornasier, M., Langer, A., Schnlieb, C.-B.: A convergent overlapping domain decomposition method for total variation minimization. Numer. Math. 116(4), 645–685 (2010)

    Article  MathSciNet  MATH  Google Scholar 

  19. Goldstein, T., Osher, S.: The split Bregman method for L1-regularized problems. SIAM J. Imaging Sci. 2(2), 323–343 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  20. Hajlaoui, N., Chaux, C., Perrin, G., Falzon, F., Benazza-Benyahia, A.: Satellite image restoration in the context of a spatially varying point spread function. J. Opt. Soc. Am. A 27(6), 1473–1481 (2010)

    Article  Google Scholar 

  21. Hansen, P.C., Nagy, J.G., O’Leary, D.P.: Deblurring Images. Matrices, Spectra, and Filtering. Society for Industrial and Applied Mathematics, Philadelphia (2006)

    Book  MATH  Google Scholar 

  22. Harmeling, S., Hirsch, M., Schölkopf, B.: Space-variant single-image blind deconvolution for removing camera shake. In: Lafferty, J.D., Williams, C.K.I., Shawe-Taylor, J., Zemel, R.S., Culotta, A. (eds.) Advances in Neural Information Processing Systems, vol. 23, pp. 829–837. Curran Associates, Inc., New York (2010)

  23. Hirsch, M., Schuler, C.J., Harmeling, S., Scholkopf, B.: Fast removal of non-uniform camera shake. In: Proceedings of the 2011 International Conference on Computer Vision, ICCV ’11, pp. 463–470. IEEE Computer Society, Washington, DC (2011)

  24. Hirsch, M., Sra, S., Scholkopf, B., Harmeling, S.: Efficient filter flow for space-variant multiframe blind deconvolution. In: 2010 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 607–614 (2010)

  25. Joshi, N., Kang, S.B., Zitnick, C.L., Szeliski, R.: Image deblurring using inertial measurement sensors. ACM Trans. Graph. 29(4), 30:1–30:9 (2010)

    Article  Google Scholar 

  26. Komodakis, N., Pesquet, J.-C.: Playing with duality: an overview of recent primal-dual approaches for solving large-scale optimization problems. arXiv:1406.5429 (2014)

  27. Kutyniok, G., Shahram, M., Zhuang, X.: Shearlab: A rational design of a digital parabolic scaling algorithm. arXiv:1106.1319 (2011)

  28. Levin, A.: Blind motion deblurring using image statistics. In NIPS, vol. 2, p. 4 (2006)

  29. Lions, P.L., Mercier, B.: Splitting algorithms for the sum of two nonlinear operators. SIAM J. Numer. Anal. 16(6), 964–979 (1979)

    Article  MathSciNet  MATH  Google Scholar 

  30. Mallat, S.: A Wavelet Tour of Signal Processing, 2nd edn. Academic Press, London (1999)

    MATH  Google Scholar 

  31. Moreau, J.J.: Proximité et dualité dans un espace hilbertien. Bull. Math. Soc. Fr. 93, 273–299 (1965)

    Article  MATH  Google Scholar 

  32. Nagy, J.G., O’Leary, D.P.: Restoring images degraded by spatially variant blur. SIAM J. Sci. Comput. 19(4), 1063–1082 (1998)

    Article  MathSciNet  MATH  Google Scholar 

  33. O’Connor, D., Vandenberghe, L.: Primal-dual decomposition by operator splitting and applications to image deblurring. SIAM J. Imaging Sci. 7, 1724–1754 (2014)

    Article  MathSciNet  MATH  Google Scholar 

  34. Combettes, Patrick L.P.L., Pesquet, J.-C.: Primal-dual splitting algorithm for solving inclusions with mixtures of composite, Lipschitzian, and parallel-sum type monotone operators. Set-Valued Var. Anal. 20(2), 307–330 (2012)

    Article  MathSciNet  MATH  Google Scholar 

  35. Preza, C., Conchello, J.-A.: Depth-variant maximum-likelihood restoration for three-dimensional fluorescence microscopy. JOSA A 21(9), 1593–1601 (2004)

    Article  Google Scholar 

  36. Pustelnik, N., Chaux, C., Pesquet, J.-C.: Parallel proximal algorithm for image restoration using hybrid regularization. IEEE Trans. Image Process. 20(9), 2450–2462 (2011)

    Article  MathSciNet  Google Scholar 

  37. Rockafellar, R.T.: Convex Analysis, 2nd edn. Princeton University Press, Princeton (1970)

    Book  MATH  Google Scholar 

  38. Rudin, L., Osher, S.J., Fatemi, E.: Nonlinear total variation based noise removal algorithms. Phys. D 60, 259–268 (1992)

    Article  MathSciNet  MATH  Google Scholar 

  39. Rudin, L.I., Osher, S.: Total variation based image restoration with free local constraints. In: Proceedings ICIP-94, IEEE International Conference on Image Processing, 1994, vol. 1, pp. 31–35 (1994)

  40. Vogel, C.R.: Computational Methods for Inverse Problems. Society for Industrial and Applied Mathematics, Philadelphia (2002)

    Book  MATH  Google Scholar 

  41. Whyte, O., Sivic, J., Zisserman, A., Ponce, J.: Non-uniform deblurring for shaken images. Int. J. Comput. Vision 98(2), 168–186 (2012)

    Article  MathSciNet  MATH  Google Scholar 

  42. Köhler, R., Hirsch, M., Mohler, B., Schölkopf, B., Harmeling, S.: Recording and playback of camera shake: benchmarking blind deconvolution with a real-world database. In: Computer Vision–ECCV 2012, pp. 27–40. Springer (2012)

  43. Candes, E., Demanet, L., Donoho, D., Ying, L.: Fast discrete curvelet transforms. Multiscale Model. Simul. 5(3), 861–899 (2006)

    Article  MathSciNet  MATH  Google Scholar 

  44. Pock, T., Cremers, D., Bischof, H., Chambolle, A.: An algorithm for minimizing the Mumford–Shah functional. In: 2009 IEEE 12th International Conference on Computer Vision, pp. 1133–1140. IEEE (2009)

Download references

Acknowledgements

Research supported in part by National Science Foundation Grants DMS-1115963 and ECCS 1509789.

Author information

Authors and Affiliations

  1. Department of Mathematics, University of California Los Angeles, Los Angeles, CA, USA

    Daniel O’Connor

  2. Electrical Engineering Department, University of California Los Angeles, Los Angeles, CA, USA

    Lieven Vandenberghe

Authors
  1. Daniel O’Connor
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Lieven Vandenberghe
    View author publications

    You can also search for this author inPubMed Google Scholar

Corresponding author

Correspondence to Daniel O’Connor.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

O’Connor, D., Vandenberghe, L. Total variation image deblurring with space-varying kernel. Comput Optim Appl 67, 521–541 (2017). https://doi.org/10.1007/s10589-017-9901-1

Download citation

  • Received:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10589-017-9901-1

Keywords

Mathematics Subject Classification