Skip to main content
SpringerLink
Log in

Deep non-blind deblurring network for saturated blurry images

  • Original Article
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

Non-blind image deblurring has attracted a lot of attention in the field of low-level vision. However, the existing non-blind deblurring methods cannot effectively deal with a saturated blurry image. The key point is that the degradation model of saturated blurry images does not satisfy the linear convolution model of a conventional blurry image. To solve the problem, in this paper, we proposed a novel deep non-blind deblurring method, dubbed saturated image non-blind deblurring network(SDBNet). The SDBNet contains two trainable sub-network, i.e., confident estimate network (CEN) and detail enhance network (DEN). Specifically, the SDBNet uses CEN to estimate the confidence map for the saturated blurry image, which is used to recognize saturated pixels in the blurry image, and then uses the confidence map, and blur kernel to restore the blurry image. Finally, we use DEN to enhance the edges and textures of the restored image. We first pre-train CEN and DEN. In order to effectively pre-train CEN, we propose a new robust function, which is used to generate label data for CEN. The experimental results show that compared with several existing non-blind deblurring methods, SDBNet can effectively restore saturated blurry images and better restore the texture, edge, and other structural information of blurry images.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Algorithm 1
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

Code availability

Not applicable.

References

  1. Cheng S, Liu R, He Y, Fan X, Luo Z (2020) Blind image deblurring via hybrid deep priors modeling. Neurocomputing 387:334–345

    Article  Google Scholar 

  2. Javaran TA, Hassanpour H, Abolghasemi V (2017) Non-blind image deconvolution using a regularization based on re-blurring process. Comput Vis Image Underst 154:16–34

    Article  Google Scholar 

  3. Wang W, Su C (2022) An optimization method for motion blur image restoration and ringing suppression via texture mapping. ISA Trans 131:650–661

    Article  Google Scholar 

  4. Richardson WH (1972) Bayesian-based iterative method of image restoration. JOSA 62(1):55–59

    Article  Google Scholar 

  5. Wiener N, Wiener N, Mathematician C, Wiener N, Wiener N, Mathématicien C (1949) Extrapolation, interpolation, and smoothing of stationary time series: with engineering applications, vol. 113, no. 21

  6. Krishnan D, Fergus R (2009) Fast image deconvolution using hyper-laplacian priors. In: NeurIPS, pp 1033–1041

  7. Chan SH, Wang X, Elgendy OA (2017) Plug-and-play ADMM for image restoration: fixed-point convergence and applications. IEEE Trans Comput 3(1):84–98

    MathSciNet  Google Scholar 

  8. Schmidt U, Rother C, Nowozin S, Jancsary J, Roth S (2013) Discriminative non-blind deblurring. In: IEEE CVPR, pp 604–611

  9. Schmidt U, Jancsary J, Nowozin S, Roth S, Rother C (2016) Cascades of regression tree fields for image restoration. IEEE Trans Pattern Anal Mach Intell 38(4):677–689

    Article  Google Scholar 

  10. Tappen MF, Liu C, Adelson EH, Freeman WT (2007) Learning gaussian conditional random fields for low-level vision. In: IEEE CVPR

  11. Zoran D, Weiss Y (2011) From learning models of natural image patches to whole image restoration. In: IEEE ICCV, pp 479–486

  12. Chen Y, Yu W, Pock T (2015) On learning optimized reaction diffusion processes for effective image restoration. In: IEEE CVPR, pp 5261–5269

  13. Chen Y, Pock T (2017) Trainable nonlinear reaction diffusion: a flexible framework for fast and effective image restoration. IEEE Trans Pattern Anal Mach Intell 39(6):1256–1272

    Article  Google Scholar 

  14. Zhang K, Zuo W, Gu S, Zhang L (2017) Learning deep CNN denoiser prior for image restoration. In: IEEE CVPR, pp 3929–3938

  15. Zhang K, Li Y, Zuo W, Zhang L, Van Gool L, Timofte R (2021) Plug-and-play image restoration with deep denoiser prior. IEEE Trans Pattern Anal Mach Intell 44(10):6360–6376

    Article  Google Scholar 

  16. Zhang J, Pan J, Lai W-S, Lau RW, Yang MH (2017) Learning fully convolutional networks for iterative non-blind deconvolution. In: IEEE CVPR, pp 3817–3825

  17. Chen L, Zhang J, Lin S, Fang F, Ren JS (2021) Blind deblurring for saturated images. In: IEEE CVPR, pp 6308–6316

  18. Cho S, Wang J, Lee S (2011) Handling outliers in non-blind image deconvolution. In: IEEE ICCV, pp 495–502

  19. Whyte O, Sivic J, Zisserman A (2014) Deblurring shaken and partially saturated images. Int J Comput Vision 110(2):185–201

    Article  Google Scholar 

  20. Pan J, Lin Z, Su Z, Yang M (2016) Robust kernel estimation with outliers handling for image deblurring. In: IEEE CVPR, pp 2800–2808

  21. Zhang X, Wang R, Chen D, Zhao Y, Gao W (2021) Handling outliers by robust m-estimation in blind image deblurring. IEEE Trans Multimed 23:3215–3226

    Article  Google Scholar 

  22. Dong J, Pan J (2021) Deep outlier handling for image deblurring. IEEE Trans Image Process 30:1799–1811

    Article  Google Scholar 

  23. Chen L, Zhang J, Pan J, Lin S, Fang F, Ren JS (2021) Learning a non-blind deblurring network for night blurry images. In: IEEE CVPR, Vol. 10, pp 542–550

  24. Pan J, Sun D, Pfister H, Yang M (2018) Deblurring images via dark channel prior. IEEE Trans Pattern Anal Mach Intell 40(10):2315–2328

    Article  Google Scholar 

  25. Chen L, Fang F, Wang T, Zhang G (2019) Blind image deblurring with local maximum gradient prior. In: IEEE CVPR, pp 1742–1750

  26. Shan Q, Jia J, Agarwala A (2008) High-quality motion deblurring from a single image. ACM Trans Graph 27(3):73

    Article  Google Scholar 

  27. Pan J, Hu Z, Su Z, Yang M (2014) Deblurring text images via l0-regularized intensity and gradient prior. In: IEEE CVPR, pp 2901–2908

  28. Pan J, Hu Z, Su Z, Yang M-H (2016) \(l_0\)-regularized intensity and gradient prior for deblurring text images and beyond. IEEE Trans Pattern Anal Mach Intell 39(2):342–355

    Article  Google Scholar 

  29. Xu L, Zheng S, Jia J (2013) Unnatural L0 sparse representation for natural image deblurring. In: IEEE CVPR, pp 1107–1114

  30. Ren W, Cao X, Pan J, Guo X, Zuo W, Yang M (2016) Image deblurring via enhanced low-rank prior. IEEE Trans Image Process 25(7):3426–3437

    Article  MathSciNet  Google Scholar 

  31. Yan Y, Ren W, Guo Y, Wang R, Cao X (2017) Image deblurring via extreme channels prior. In: IEEE CVPR, pp 6978–6986

  32. Rudin LI, Osher SJ (1994) Total variation based image restoration with free local constraints. In: IEEE ICIP, pp 31–35

  33. Wang Y, Yang J, Yin W, Zhang Y (2008) A new alternating minimization algorithm for total variation image reconstruction. SIAM J Imag Sci 1(3):248–272

    Article  MathSciNet  Google Scholar 

  34. Fortunato HE, Oliveira MM (2014) Fast high-quality non-blind deconvolution using sparse adaptive priors. Visual Comput 30(6–8):661–671

    Article  Google Scholar 

  35. Schmidt U, Gao Q, Roth S (2010) A generative perspective on MRFS in low-level vision. In: IEEE CVPR, pp. 1751–1758

  36. Schmidt U, Roth S (2014) Shrinkage fields for effective image restoration. In: IEEE CVPR, pp 2774–2781

  37. Li L, Pan J, Lai W, Gao C, Sang N, Yang M (2019) Blind image deblurring via deep discriminative priors. Int J Comput Vision 127(8):1025–1043

    Article  Google Scholar 

  38. Zhang J, Pan J, Lai W, Lau RWH, Yang M (2017) Learning fully convolutional networks for iterative non-blind deconvolution. In: IEEE CVPR, pp 6969–6977

  39. Gong D, Zhang Z, Shi Q, van den Hengel A, Shen C, Zhang Y (2020) Learning deep gradient descent optimization for image deconvolution. IEEE Trans Neural Netw Learn Syst 31(12):5468–5482

    Article  MathSciNet  Google Scholar 

  40. Dong J, Roth S, Schiele B (2021) Learning spatially-variant MAP models for non-blind image deblurring. In: IEEE CVPR, pp 4886–4895

  41. Wang D, Tang H, Pan J, Tang J (2021) Learning a tree-structured channel-wise refinement network for efficient image deraining. In: IEEE ICME, pp 1–6

  42. Wang D, Pan J, Tang J (2023) Single image deraining using residual channel attention networks. J Comput Sci Technol 38(2):439–454

    Article  Google Scholar 

  43. Dong J, Pan J, Su Z, Yang M (2017) Blind image deblurring with outlier handling. In: IEEE ICCV, pp 2497–2505

  44. Hu Z, Cho S, Wang J, Yang M (2014) Deblurring low-light images with light streaks. In: IEEE CVPR, pp 3382–3389

  45. Wang D, Liu J, Liu R, Fan X (2023) An interactively reinforced paradigm for joint infrared-visible image fusion and saliency object detection. Inf Fus 98:101828

    Article  Google Scholar 

  46. Zhang Y, Li K, Li K, Wang L, Zhong B, Fu Y (2018) Image super-resolution using very deep residual channel attention networks. In: ECCV, vol. 11211, pp 294–310

  47. Levin A, Weiss Y, Durand F, Freeman W (2009) Understanding and evaluating blind deconvolution algorithms. In: IEEE CVPR, pp 1964–1971

  48. Dabov K, Foi A, Katkovnik V, Egiazarian KO (2008) Image restoration by sparse 3d transform-domain collaborative filtering. In: SPIE, vol. 6812, pp 681207

  49. Schuler CJ, Burger HC, Harmeling S, Schölkopf B (2013) A machine learning approach for non-blind image deconvolution. In: IEEE CVPR, pp 1067–1074

  50. Son H, Lee S (2017) Fast non-blind deconvolution via regularized residual networks with long/short skip-connections. In: IEEE ICCP, pp 23–32

  51. Liu Y, Lai W, Chen Y, Kao Y, Yang M, Chuang Y, Huang J (2020) Single-image HDR reconstruction by learning to reverse the camera pipeline. In: IEEE CVPR, pp 1648–1657

  52. Gharbi M, Chen J, Barron JT, Hasinoff SW, Durand F (2017) Deep bilateral learning for real-time image enhancement. ACM Trans Graph 36(4): 1–12

  53. Ma L, Ma T, Liu R, Fan X, Luo Z (2022) Toward fast, flexible, and robust low-light image enhancement. In: IEEE CVPR, pp 5627–5636

  54. Mildenhall B, Srinivasan PP, Tancik M, Barron JT, Ramamoorthi R, Ng R (2020) Nerf: representing scenes as neural radiance fields for view synthesis. In: Springer ECCV, vol. 12346, pp 405–421

  55. Zheng H, Yong H, Zhang L (2021) Deep convolutional dictionary learning for image denoising. In: IEEE CVPR, pp 630–641

  56. Zhang K, Gool LV, Timofte R (2020) Deep unfolding network for image super-resolution. In: IEEE CVPR, pp 3214–3223

Download references

Acknowledgements

This work is supported by the General project of Liaoning Provincial Department of Education, China, No. LJKZ0986; Postdoctoral Science Foundation, No. 2019M651123; Science and Technology Innovation Fund (Youth Science and Technology Star) of Dalian, China, No. 2018RQ65. Fund receiver: Dr. Bo Fu. This work is supported by the National Natural Science Foundation of China (NSFC) Grant No.61976109, China; Liaoning Provincial Key Laboratory Special Fund; Dalian Key Laboratory Special Fund. Fund receiver: Dr. Yonggong Ren. This research was funded by the University of Economics Ho Chi Minh City, Vietnam. Fund receiver: Dr. Dang Ngoc Hoang Thanh.

Author information

Authors and Affiliations

  1. Liaoning Normal University, Dalian, Liaoning, China

    Bo Fu, Shilin Fu, Yuechu Wu, Yuanxin Mao & Yonggong Ren

  2. University of Economics Ho Chi Minh City, Ho Chi Minh City, Vietnam

    Dang N. H. Thanh

Authors
  1. Bo Fu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Shilin Fu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yuechu Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yuanxin Mao
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Yonggong Ren
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Dang N. H. Thanh
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Dang N. H. Thanh.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Ethical approval

This article does not contain any studies with human participants or animals performed by any of the authors.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Fu, B., Fu, S., Wu, Y. et al. Deep non-blind deblurring network for saturated blurry images. Neural Comput & Applic 36, 7829–7843 (2024). https://doi.org/10.1007/s00521-024-09495-3

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-024-09495-3

Keywords

Search

Navigation