Skip to main content
SpringerLink
Log in

Reference-guided deep deblurring via a selective attention network

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

Image deblurring is an important problem encountered in many image restoration tasks. To remove the motion blur of images captured from dynamic scenes, various Convolutional Neural Networks (CNNs) based methods are developed to restore the latent sharp image via an end-to-end trainable. However, these CNNs-based methods cannot restore enough structure details as no significant information is provided by the blurry input only. In this work, we propose a reference-guided deep deblurring method by incorporating the high-quality reference image into the deep network for better deblurring effect. Concretely, the correlation between the blurry input and the reference image is computed in the high-level feature space, and further represented by the similarity maps. To pick up the most relevant similarity maps to the input, the selective attention module is designed, and the selected similarity maps are decoded to reconstruct the deblurred sharp image. Extensive experimental results on two public datasets demonstrate that the proposed method achieves superior performance to existing methods quantitatively and qualitatively. More ablation experiments also validate that the favourable deblurred results can still be obtained even if the reference image is not similar with the input image.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

References

  1. Bai Y, et al. (2018) Graph-based blind image deblurring from a single photograph. IEEE Trans Image Proc 28.3:1404–1418

    MathSciNet  Google Scholar 

  2. Chakrabarti A (2016) A neural approach to blind motion deblurring. In: ECCV

  3. Chen X, Yang J, Wu Q (2010) Image deblur in gradient domain. Opt Eng 49 :117003

    Article  Google Scholar 

  4. Cho S, Lee S (2009) Fast motion deblurring. In: SIG- GRAPH 2009

  5. Gao H et al (2019) Dynamic scene deblurring with parameter selective sharing and nested skip connections. In: 2019 IEEE/CVF Con- ference on computer vision and pattern recognition (CVPR), pp 3843–3851

  6. He K et al (2016) Deep residual learning for image recognition. In: 2016 IEEE Conference on computer vision and pattern recognition (CVPR), pp 770–778

  7. Jia J (2014) Mathematical models and practical solvers for uniform motion deblurring. In: Motion Deblurring

  8. Jia J (2007) Single image motion deblurring using transparency. In: 2007 IEEE Conference on computer vision and pattern recognition, pp 1–8

  9. Jolicoeur-Martineau A (2019) The relativistic discriminator: a key element missing from standard GAN. arXiv:1807.00734

  10. Kim T, Mu Lee K (2015) Generalized video deblurring for dynamic scenes. In: 2015 IEEE Conference on computer vision and pattern recognition (CVPR), pp 5426–5434

  11. Kingma DP, Ba J (2014) Adam: A method for stochastic optimization. arXiv:1412.6980

  12. Krishnan D, Fergus R (2009) Fast Image Deconvolution using Hyper-Laplacian Priors. In: NIPS

  13. Kupyn O, et al. (2019) DeblurGAN-v2: deblurring (orders-of-magnitude) faster and better. In: 2019 IEEE/CVF international conference on computer vision (ICCV), pp 8877–8886

  14. Kupyn O et al (2018) DeblurGAN: Blind motion deblurring using conditional adversarial networks. In: 2018 IEEE/CVF conference on computer vision and pattern recognition, pp 8183–8192

  15. Lai W-S et al (2016) A comparative study for single image blind deblurring. In: 2016 IEEE conference on computer vision and pattern recognition (CVPR), pp 1701–1709

  16. Lai W-S et al (2016) A comparative study for single image blind deblurring. In: 2016 IEEE conference on computer vision and pattern recognition (CVPR), pp 1701–1709

  17. Levin A (2006) Blind motion deblurring using image statistics. In: NIPS

  18. Xinwang L et al (2019) Multiple kernel k k-means with incomplete kernels. IEEE Trans Pattern Anal Mach Intell 42.5:1191–1204

    Google Scholar 

  19. Michaeli T, Irani M (2014) Blind deblurring using internal patch recurrence. In: European conference on computer vision. Springer, pp 783–798

  20. Nah S, Kim T, Mu Lee K (2017) Deep multi-scale convolutional neural network for dynamic scene deblurring. In: 2017 IEEE Conference on computer vision and pattern recognition (CVPR), pp 257–265

  21. Pan J, et al. (2016) Blind image deblurring using dark channel prior. In: 2016 IEEE conference on computer vision and pattern recognition (CVPR), pp 1628–1636

  22. Paszke A et al (2017) Automatic differentiation in pytorch

  23. Rajagopalan A, Chellappa R (2014) Motion Deblurring: algorithms and systems

  24. Ren D et al (2020) Neural blind deconvolution using deep priors. In: 2020 IEEE/CVF Conference on computer vision and pattern recognition (CVPR), pp 3338–3347

  25. Sellent A, Rother C, Roth S (2016) Stereo video deblurring. arXiv:1607.08421

  26. Shan Q, Jia J, Agarwala A (2008) High-quality motion deblurring from a single image. ACM Trans Graph 27:73

    Google Scholar 

  27. Shim G, Park J, Kweon IS (2020) Robust reference-based super-resolution with similarity-aware deformable convolution. In: 2020 IEEE/CVF conference on computer vision and pattern recognition (CVPR), pp 8422–8431

  28. Sun J et al (2015) Learning a convolutional neural network for non-uniform motion blur removal. In: 2015 IEEE Conference on computer vision and pattern recognition (CVPR), pp 769– 777

  29. Tao X et al (2018) Scale-recurrent network for deep image deblurring. In: 2018 IEEE/CVF Conference on computer vision and pattern recognition, pp 8174–8182

  30. Wang Y et al (2017) The light field attachment: Turning a DSLR into a light field camera using a low budget camera ring. IEEE Tran Vis Comput Graph 23(10):2357–2364

    Article  Google Scholar 

  31. Xu L, Jia J (2010) Two-phase kernel estimation for robust motion deblurring. In: ECCV

  32. Xu L et al (2014) Deep convolutional neural network for image deconvolution. In: NIPS

  33. Yan Y et al (2017) Image deblurring via extreme channels prior. In: 2017 IEEE Conference on computer vision and pattern recognition (CVPR), pp 6978–6986

  34. Yang F et al (2020) Learning texture transformer network for image super-resolution. In: 2020 IEEE/CVF conference on computer vision and pattern recognition (CVPR), pp 5790– 5799

  35. Yang F et al (2020) Learning texture transformer network for image super-resolution. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 5791–5800

  36. Yu X, Ye X, Gao Q (2020) Infrared handprint image restoration algorithm based on apoptotic mechanism. IEEE Access 8:47334–47343

    Article  Google Scholar 

  37. Zamir SW et al (2021) Multi-stage progressive image restoration. arXiv:2102.02808

  38. Zhang J et al (2018) Dynamic scene deblurring using spatially variant recurrent neural networks. In: 2018 IEEE/CVF Conference on computer vision and pattern recognition, pp 2521– 2529

  39. Zhang Z et al (2019) Image super-resolution by neural texture transfer. In: 2019 IEEE/CVF Conference on computer vision and pattern recognition (CVPR), pp 7974–7983

  40. Zhang Z et al (2019) Image super-resolution by neural texture transfer. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 7982–7991

  41. Zheng H et al (2017) Combining exemplar-based approach and learning- based approach for light field super-resolution using a hybrid imaging system. In: 2017 IEEE International conference on computer vision workshops (ICCVW), pp 2481–2486

  42. Zheng H et al (2018) CrossNet: An end-to-end reference-based super resolution network using cross-scale warping. In: ECCV

  43. Zheng H et al (2018) Crossnet: An end-to-end reference-based super resolution network using cross-scale warping. In: Proceedings of the european conference on computer vision (ECCV), pp 88–104

  44. Zheng H et al (2017) Learning cross-scale correspondence and patchbased synthesis for reference-based super-resolution. In: BMVC

  45. Zhou S et al (2020) Cross-scale internal graph neural network for image super-resolution. In: Advances in neural information processing systems 33

Download references

Acknowledgements

This work was supported by the General Program of National Natural Science Foundation of China (NSFC) (Grant No. 61806147), and Shanghai Natural Science Foundation of China (Grant No. 18ZR1441200).

Author information

Authors and Affiliations

  1. Tongji University, Shanghai, China

    Yaowei Li, Ye Luo & Jianwei Lu

Authors
  1. Yaowei Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ye Luo
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jianwei Lu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Ye Luo or Jianwei Lu.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Li, Y., Luo, Y. & Lu, J. Reference-guided deep deblurring via a selective attention network. Appl Intell 52, 3867–3879 (2022). https://doi.org/10.1007/s10489-021-02585-y

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-021-02585-y

Keywords

Search

Navigation