Skip to main content

Advertisement

Springer Nature Link
Log in

CDMC-Net: Context-Aware Image Deblurring Using a Multi-scale Cascaded Network

  • Published:
Neural Processing Letters Aims and scope Submit manuscript

Abstract

Image deblurring is a widely researched topic in low-level vision. Over the last few years, many researchers try to deblur by stacking multi-scale pyramid structures, which inevitably increases the computational complexity. In addition, most of the existing deblurring methods do not adequately model long-range contextual information, making the structure of blurred objects not well restored. To address the above issues, we propose a novel context-aware multi-scale convolutional neural network (CDMC-Net) for image deblurring. We progressively restore latent sharp images in two stages, and a cross-stage feature aggregation (CSFA) strategy is introduced to enhance the information flow interaction between the two stages. The key design of CDMC-Net to reduce the complexity is the use of a multi-input multi-output encoder-decoder at each stage, which can process multi-scale blurry images in a coarse-to-fine manner. Furthermore, to effectively capture long-range context information in different scenarios, we propose a multi-strip feature extraction module (MSFM). Its strip pooling with different kernel sizes allows the network to aggregate rich global and local contextual information. Extensive experimental results demonstrate that CDMC-Net outperforms state-of-the-art motion deblurring methods on both synthetic benchmark datasets and real blurred images. We also use CDMC-Net as a pre-processing step for object detection to further verify the effectiveness of our proposed deblurring method in downstream vision tasks.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13

Similar content being viewed by others

Explore related subjects

Discover the latest articles and news from researchers in related subjects, suggested using machine learning.

Availability of data or code

Code and data are available.

References

  1. Mao L, Li X, Yang D, Zhang R (2021) Convolutional feature frequency adaptive fusion object detection network. Neural Process Lett 53(5):3545–3560

    Article  Google Scholar 

  2. Leng J, Liu Y, Wang Z, Hu H, Gao X (2022) Crossnet: detecting objects as crosses. IEEE Trans Multimedia 24:861–875

    Article  Google Scholar 

  3. Ke R, Aviles-Rivero AI, Pandey S, Reddy S, Schönlieb C-B (2022) A three-stage self-training framework for semi-supervised semantic segmentation. IEEE Trans Image Process 31:1805–1815

    Article  Google Scholar 

  4. Li Y, Li M, Li Z, Xiao C, Li H (2022) Efrnet: Efficient feature reuse network for real-time semantic segmentation. Neural Processing Letters 1–13. https://doi.org/10.1007/s11063-022-10740-w

  5. Vaquero L, Brea VM, Mucientes M (2022) Tracking more than 100 arbitrary objects at 25 fps through deep learning. Pattern Recogn 121:108205

    Article  Google Scholar 

  6. Xiang X, Ren W, Qiu Y, Zhang K, Lv N (2021) Multi-object tracking method based on efficient channel attention and switchable atrous convolution. Neural Process Lett 53(4):2747–2763

    Article  Google Scholar 

  7. Tang S, Xie X, Xia M, Luo L, Liu P, Li Z (2018) Spatial-scale-regularized blur kernel estimation for blind image deblurring. Signal Processing: Image Communication 68:138–154

    Google Scholar 

  8. Chen L, Fang F, Wang T, Zhang G (2019) Blind image deblurring with local maximum gradient prior. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 1742–1750

  9. Krishnan D, Tay T, Fergus R (2011) Blind deconvolution using a normalized sparsity measure. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 233–240. IEEE

  10. Bai Y, Cheung G, Liu X, Gao W (2018) Graph-based blind image deblurring from a single photograph. IEEE Trans Image Process 28(3):1404–1418

    Article  MathSciNet  Google Scholar 

  11. Xu L, Zheng S, Jia J (2013) Unnatural l0 sparse representation for natural image deblurring. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1107–1114

  12. Pan J, Sun D, Pfister H, Yang M-H (2016) Blind image deblurring using dark channel prior. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1628–1636

  13. Bochkovskiy A, Wang C-Y, Liao H-YM (2020) Yolov4: Optimal speed and accuracy of object detection. arXiv preprint arXiv:2004.10934

  14. Liu J, Shao H, Jiang Y, Deng X (2022) Cnn-based hidden-layer topological structure design and optimization methods for image classification. Neural Processing Letters 1–12. https://doi.org/10.1007/s11063-022-10742-8

  15. Sun J, Cao W, Xu Z, Ponce J (2015) Learning a convolutional neural network for non-uniform motion blur removal. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 769–777

  16. Schuler CJ, Hirsch M, Harmeling S, Schölkopf B (2015) Learning to deblur. IEEE Trans Pattern Anal Mach Intell 38(7):1439–1451

    Article  Google Scholar 

  17. Xu X, Pan J, Zhang Y-J, Yang M-H (2017) Motion blur kernel estimation via deep learning. IEEE Trans Image Process 27(1):194–205

    Article  MathSciNet  MATH  Google Scholar 

  18. Nah S, Hyun Kim T, Mu Lee K (2017) Deep multi-scale convolutional neural network for dynamic scene deblurring. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3883–3891

  19. Purohit K, Rajagopalan A (2020) Region-adaptive dense network for efficient motion deblurring. Proceedings of the AAAI Conference on Artificial Intelligence 34:11882–11889

    Article  Google Scholar 

  20. Zhang K, Luo W, Zhong Y, Ma L, Stenger B, Liu W, Li H (2020) Deblurring by realistic blurring. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 2737–2746

  21. Chen C, Chen Q, Xu J, Koltun V (2018) Learning to see in the dark. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3291–3300

  22. Cai J, Zuo W, Zhang L (2020) Dark and bright channel prior embedded network for dynamic scene deblurring. IEEE Trans Image Process 29:6885–6897

    Article  MATH  Google Scholar 

  23. Tang K, Xu D, Liu H, Zeng Z (2021) Context module based multi-patch hierarchical network for motion deblurring. Neural Process Lett 53(1):211–226

    Article  Google Scholar 

  24. Shen Z, Wang W, Lu X, Shen J, Ling H, Xu T, Shao L (2019) Human-aware motion deblurring. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 5572–5581

  25. Suin M, Purohit K, Rajagopalan A (2020) Spatially-attentive patch-hierarchical network for adaptive motion deblurring. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3606–3615

  26. Zhang Y, Li K, Li K, Zhong B, Fu Y (2019) Residual non-local attention networks for image restoration. arXiv preprint arXiv:1903.10082

  27. Kupyn O, Budzan V, Mykhailych M, Mishkin D, Matas J (2018) Deblurgan: Blind motion deblurring using conditional adversarial networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 8183–8192

  28. Kupyn O, Martyniuk T, Wu J, Wang Z (2019) Deblurgan-v2: Deblurring (orders-of-magnitude) faster and better. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 8878–8887

  29. Tao X, Gao H, Shen X, Wang J, Jia J (2018) Scale-recurrent network for deep image deblurring. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 8174–8182

  30. Gao H, Tao X, Shen X, Jia J (2019) Dynamic scene deblurring with parameter selective sharing and nested skip connections. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3848–3856

  31. Wan S, Tang S, Xie X, Gu J, Huang R, Ma B, Luo L (2021) Deep convolutional-neural-network-based channel attention for single image dynamic scene blind deblurring. IEEE Trans Circuits Syst Video Technol 31(8):2994–3009

    Article  Google Scholar 

  32. Wu Y, Hong C, Zhang X, He Y (2021) Stack-based scale-recurrent network for face image deblurring. Neural Process Lett 53(6):4419–4436

    Article  Google Scholar 

  33. Pan J, Bai H, Tang J.: Cascaded deep video deblurring using temporal sharpness prior. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3043–3051 (2020)

  34. Zamir SW, Arora A, Khan S, Hayat M, Khan FS, Yang M-H, Shao L (2021) Multi-stage progressive image restoration. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 14821–14831

  35. Chen L, Lu X, Zhang J, Chu X, Chen C (2021) Hinet: Half instance normalization network for image restoration. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 182–192

  36. Zhang D, Shao J, Liang Z, Gao L, Shen HT (2021) Large factor image super-resolution with cascaded convolutional neural networks. IEEE Trans Multimedia 23:2172–2184

    Article  Google Scholar 

  37. Hou Q, Zhang L, Cheng M-M, Feng J (2020) Strip pooling: Rethinking spatial pooling for scene parsing. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4003–4012

  38. Shan Q, Jia J, Agarwala A (2008) High-quality motion deblurring from a single image. Acm transactions on graphics (tog) 27(3):1–10

    Article  Google Scholar 

  39. Whyte O, Sivic J, Zisserman A, Ponce J (2012) Non-uniform deblurring for shaken images. Int J Comput Vision 98(2):168–186

    Article  MathSciNet  MATH  Google Scholar 

  40. Bai Y, Jia H, Jiang M, Liu X, Xie X, Gao W (2020) Single-image blind deblurring using multi-scale latent structure prior. IEEE Trans Circuits Syst Video Technol 30(7):2033–2045

    Google Scholar 

  41. Yan R, Shao L (2016) Blind image blur estimation via deep learning. IEEE Trans Image Process 25(4):1910–1921

    MathSciNet  MATH  Google Scholar 

  42. Gong D, Yang J, Liu L, Zhang Y, Reid I, Shen C, Van Den Hengel A, Shi Q (2017) From motion blur to motion flow: A deep learning solution for removing heterogeneous motion blur. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2319–2328

  43. Lin T-Y, Dollár P, Girshick R, He K, Hariharan B, Belongie S (2017) Feature pyramid networks for object detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2117–2125

  44. Zhang H, Dai Y, Li H, Koniusz P (2019) Deep stacked hierarchical multi-patch network for image deblurring. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 5978–5986

  45. Park D, Kang DU, Kim J, Chun SY (2020) Multi-temporal recurrent neural networks for progressive non-uniform single image deblurring with incremental temporal training. In: European Conference on Computer Vision, pp. 327–343. Springer

  46. Cho S-J, Ji S-W, Hong J-P, Jung S-W, Ko S-J (2021) Rethinking coarse-to-fine approach in single image deblurring. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 4641–4650

  47. Tsai F-J, Peng Y-T, Lin Y-Y, Tsai C-C, Lin C-W (2021) Banet: Blur-aware attention networks for dynamic scene deblurring. arXiv e-prints, 2101 arXiv:2101.07518

  48. Guo C, Li C, Guo J, Cong R, Fu H, Han P (2018) Hierarchical features driven residual learning for depth map super-resolution. IEEE Trans Image Process 28(5):2545–2557

    Article  MathSciNet  Google Scholar 

  49. Pang Y, Wang T, Anwer RM, Khan FS, Shao L (2019) Efficient featurized image pyramid network for single shot detector. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 7336–7344

  50. Jiang K, Wang Z, Yi P, Chen C, Wang G, Han Z, Jiang J, Xiong Z (2021) Multi-scale hybrid fusion network for single image deraining. IEEE Transactions on Neural Networks and Learning Systems. https://doi.org/10.1109/TNNLS.2021.3112235

    Article  Google Scholar 

  51. Rim J, Lee H, Won J, Cho S (2020) Real-world blur dataset for learning and benchmarking deblurring algorithms. In: European Conference on Computer Vision, pp. 184–201. Springer

  52. Huynh-Thu Q, Ghanbari M (2008) Scope of validity of psnr in image/video quality assessment. Electron Lett 44(13):800–801

    Article  Google Scholar 

  53. Wang Z, Bovik AC, Sheikh HR, Simoncelli EP (2004) Image quality assessment: from error visibility to structural similarity. IEEE Trans Image Process 13(4):600–612

    Article  Google Scholar 

Download references

Funding

This work was supported by the National Natural Science Foundation of China under Grants 62066047, 61966037.

Author information

Authors and Affiliations

  1. School of Information Science and Engineering, Yunnan University, Kunming, 650504, China

    Qian Zhao, Dongming Zhou & Hao Yang

Authors
  1. Qian Zhao
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Dongming Zhou
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Hao Yang
    View author publications

    You can also search for this author inPubMed Google Scholar

Contributions

Zhao Qian and Yang Hao conceived the experiments, and Zhao Qian performed the experiments. All authors reviewed the manuscript.

Corresponding author

Correspondence to Dongming Zhou.

Ethics declarations

Conflicts of interest

The authors declared that they have no conflicts of interest to this work.

Ethical approval

The experiments in this article are all realized through program operation, which will not cause harm to humans and animals, and will not cause moral and ethical problems.

Consent to participate:

Welcome readers to communicate.

Consent for publication:

Completed at Yunnan University on March 22, 2022

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhao, Q., Zhou, D. & Yang, H. CDMC-Net: Context-Aware Image Deblurring Using a Multi-scale Cascaded Network. Neural Process Lett 55, 3985–4006 (2023). https://doi.org/10.1007/s11063-022-10976-6

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11063-022-10976-6

Keywords