Skip to main content
SpringerLink
Log in

Revisiting reweighted graph total variation blind deconvolution and beyond

  • Original article
  • Published:
The Visual Computer Aims and scope Submit manuscript

Abstract

It is known that image priors are essential to blind deconvolution. Reweighted graph total variation (RGTV), as a new prior to substitute the most classical total variation (TV), has been shown superior to TV and several other cutting-edge models both theoretically and empirically. This paper steps forward, firstly providing a simpler geometric perspective to RGTV in the framework of variational partial differential equations (PDEs), rather than the previous graph spectral interpretation made in the graph frequency domain. Surprisingly, a slight shift of perspective as such finally leads to a huge blind deblurring performance boosting in both accuracy and efficiency as compared to the previously derived numerical approach, which approximates RGTV as the graph L1-Laplacian regularizer. Utilizing the simplified RGTV as reformulated in this paper, another valuable contribution is an exploration of its potentials for blind facial image restoration by combining unsupervised deep facial models. Experimental results of blind face deblurring and blind face hallucination both demonstrate necessity and rationale of a joint model-based and learning-based approach to blind face restoration.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig.1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

Data availability

The data that support the findings of this study are partly available from https://github.com/continue-cmd/results.

References

  1. Eqtedaei, A., Ahmadyfard, A.: Coarse-to-fine blind image deblurring based on K-means clustering. Vis. Comput. 12, 36–415 (2023). https://doi.org/10.1007/s00371-023-02785-2

    Article  Google Scholar 

  2. Parvaz, R.: Point spread function estimation for blind image deblurring problems based on framelet transform. Vis. Comput. (2022). https://doi.org/10.1007/s00371-022-02484-4

    Article  Google Scholar 

  3. Lai, W.S., Huang, J.B., Hu, Z., Ahuja, N., Yang, M.H.: A comparative study for single image blind deblurring. In: CVPR, pp. 1701–1709 (2016)

  4. Pan, J., Hu, Z., Su, Z., Yang, M.-H.: L0-regularized intensity and gradient prior for deblurring text images and beyond. IEEE Trans. PAMI 39(2), 342–355 (2017)

    Article  Google Scholar 

  5. Pan, J., Sun, D., Pfister, H., Yang, M.H.: Deblurring images via dark channel prior. In: IEEE Transactions PAMI, pp. 2315–2328 (2018)

  6. Yan, Y., Ren, W., Guo, Y., Wang, R., Cao, X.: Image deblurring via extreme channels prior. In: CVPR (2017)

  7. Li, L., Pan, J., Lai, W.-S., Gao, C., Sang, N., Yang, M.-H.: Blind image deblurring via deep discriminative priors. Int. J. Comput. Vis. 127(8), 1025–1043 (2019)

    Article  Google Scholar 

  8. Wen, F., Ying, R., Liu, Y., Liu, P., Truong, T.-K.: A simple local minimal intensity prior and an improved algorithm for blind image deblurring. In: IEEE Transactions Circuits and Systems for Video Technology, pp. 2923–2937 (2021)

  9. Chen, L., Fang, F., Wang, T., Zhang, G.: Blind image deblurring with local maximum gradient prior. In: CVPR, pp. 1742–1750 (2019)

  10. Xu, Z., Chen, H., Li, Z.: Fast blind deconvolution using a deeper sparse patch-wise maximum gradient prior. Signal Process.: Image Commun. 90, 116050 (2021)

    Google Scholar 

  11. Levin, A., Weiss, Y., Durand, F., Freeman, W.T.: Understanding blind deconvolution algorithms. IEEE Trans. PAMI 33(12), 2354–2367 (2011)

    Article  Google Scholar 

  12. Pan, J., Su, Z.: Fast L0-regularized kernel estimation for robust motion deblurring. IEEE Signal Process. Lett. 20(9), 1107–1114 (2013)

    Google Scholar 

  13. Xu, L., Zheng, S., Jia, J.: Unnatural L0 sparse representation for natural image deblurring. In: IEEE Conferences of Computer Vision and Pattern Recognition, pp. 1107–1114 (2013)

  14. Shearer, P., Gilbert, A.C., Hero III, A.O.: Correcting camera shake by incremental sparse approximation. In: ICIP pp. 572–576 (2013)

  15. Krishnan, D., Tay, T., Fergus, R.: Blind deconvolution using a normalized sparsity measure. In: CVPR pp. 233–240 (2011)

  16. Bai, Y., Cheung, G., Liu, X., Gao, W.: Graph-based blind image deblurring from a single photograph. IEEE TIP 28(3), 1404–1418 (2019)

    MathSciNet  Google Scholar 

  17. Chan, T.F., Wong, C.K.: Total variation blind deconvolution. IEEE TIP 7(3), 370–375 (1998)

    Google Scholar 

  18. Cho, S., Lee, S.: Fast motion deblurring. ACM Trans. Graph. 28(5), 1–8 (2009)

    Article  Google Scholar 

  19. Xu, L., Jia, J.: Two-phase Kernel estimation for robust motion deblurring. In: ECCV, pp. 157–170 (2010)

  20. Sapiro, G.: Geometric Partial Differential Equations and Image Analysis. Cambridge University Press, Cambridge (2001)

    Book  Google Scholar 

  21. Karras, T., Aila, T., Laine, S., Lehtinen, J.: Progressive growing of GANs for improved quality, stability, and variation. In: ICLR (2018).

  22. Karras, T., Laine, S., Aila, T.: A style-based generator architecture for generative adversarial networks. In: CVPR (2019)

  23. Elmoataz, A., Lézoray, O., Bougleux, S.: Nonlocal discrete regularization on weighted graphs: a framework for image and manifold processing. IEEE Trans. Image Process. 17(7), 1047–1060 (2008)

    Article  MathSciNet  Google Scholar 

  24. Hidane, M., Lézoray, O., Elmoataz, A.: Nonlinear multilayered representation of graph-signals. J. Math. Imaging Vis. 45(2), 114–137 (2013)

    Article  MathSciNet  Google Scholar 

  25. Couprie, C., Grady, L., Najman, L., Pesquet, J.-C., Talbot, H.: Dual constrained TV-based regularization on graphs. SIAM J. Imag. Sci. 6(3), 1246–1273 (2013)

    Article  MathSciNet  Google Scholar 

  26. Berger, P., Hannak, G., Matz, G.: Graph signal recovery via primal dual algorithms for total variation minimization. IEEE J. Sel. Topics Signal Process. 11(6), 842–855 (2017)

    Article  Google Scholar 

  27. Shuman, D.I., Narang, S.K., Frossard, P., Ortega, A., Vandergheynst, P.: The emerging field of signal processing on graphs: extending high-dimensional data analysis to networks and other irregular domains. IEEE Signal Process. Mag. 30(3), 83–98 (2013)

    Article  Google Scholar 

  28. Pang, J., Cheung, G.: Graph Laplacian regularization for image denoising: analysis in the continuous domain. IEEE Trans. Image Process. 26(4), 1770–1785 (2017)

    Article  MathSciNet  Google Scholar 

  29. Huber, P.J., Ronchetti, E.M.: Robust Statistics, 2nd edn. Wiley series in probability and statistics (2009)

  30. Aubert, G., Kornprobst, P.: Mathematical Problems in Image Processing. Springer, New York (2002)

    Book  Google Scholar 

  31. Ren, D., Zhang, K., Wang, Q., Hu, Q., Zuo, W.: Neural blind deconvolution using deep priors. In: CVPR (2020)

  32. Yang, T., Ren, P., Xie, X., Zhang, L.: GAN prior embedded network for blind face restoration in the wild. In: CVPR (2021)

  33. Wang, X., Li, Y., Zhang, H., Shan, Y.: GFP-GAN: towards real-world blind face restoration with generative facial prior. In: CVPR (2021)

  34. Chen, C., Li, X., Yang, L., Lin, X., Wong, K., Y.K.: Progressive semantic-aware style transformation for blind face restoration. In: CVPR (2021)

  35. Bulat, A., Yang, J., Tzimiropoulos, G.: To learn image super-resolution, use a GAN to learn how to do image degradation first. In: ECCV (2018)

  36. Asim, M., Shamshad, F., Ahmed, A.: Blind Image Deconvolution Using Deep Generative Priors. IEEE Trans. Comput. Imaging 6, 1493–1506 (2020)

    Article  MathSciNet  Google Scholar 

  37. Boracchi, G., Foi, A., et al.: Modeling the performance of image restoration from motion Blur. IEEE Trans. Image Process. 21(8), 3502–3517 (2012)

    Article  MathSciNet  Google Scholar 

  38. Kingma, D.P. Ba, J.: Adam: a method for stochastic optimization. In: ICLR (2015)

  39. Schuler, C., Hirsch, M., Harmeling, S., Scholkopf, B.: Learning to Deblur. IEEE Trans. Pattern Anal. Mach. Intell. 38(7), 1439–1451 (2016)

    Article  Google Scholar 

  40. Dong, J., Pan, J., Su, Z., Yang, M.-H.: Blind image deblurring with outlier handling. In: ICCV (2017)

  41. Pan, J., Dong, J., Tai, Y.-W., Su, Z., Yang, M.-H.: Learning discriminative data fitting functions for blind image deblurring. In: ICCV (2017)

Download references

Acknowledgements

The work was supported in part by the Natural Science Foundation of China (61771250, 61972213, 11901299) and in part by the Fundamental Research Funds for the Central Universities (30918014108).

Author information

Authors and Affiliations

  1. College of Telecommunications and Information Engineering, Nanjing University of Posts and Telecommunications, Nanjing, 210003, China

    Wen-Ze Shao, Wei-Wei Luo, Jin-Ye Li & Mei-Lin Liu

  2. School of Data Science and Statistics, Nanjing Audit University, Nanjing, 211815, China

    Hai-Song Deng

Authors
  1. Wen-Ze Shao
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Hai-Song Deng
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Wei-Wei Luo
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jin-Ye Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Mei-Lin Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Wen-Ze Shao.

Ethics declarations

Conflict of interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Shao, WZ., Deng, HS., Luo, WW. et al. Revisiting reweighted graph total variation blind deconvolution and beyond. Vis Comput 40, 3119–3135 (2024). https://doi.org/10.1007/s00371-023-03014-6

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00371-023-03014-6

Keywords

Search

Navigation