Skip to main content
SpringerLink
Log in

Context Module Based Multi-patch Hierarchical Network for Motion Deblurring

  • Published:
Neural Processing Letters Aims and scope Submit manuscript

Abstract

Single image blind motion deblurring refers to transferring a blurred motion image into a corresponding clear image, which is a challenging and classic problem in the field of computer vision. The spatially variant blur is usually caused by many factors, such as camera jitter and object motion. Since image deblurring can be regarded as the task of image transformation, deep learning methods based on coarse-to-fine scheme, especially those using multi-scale architectures become popular. However, they have the disadvantages of unsatisfactory image quality and time-consuming running caused by large kernel size. In this paper, we propose a novel end-to-end network structure based on Deep Hierarchical Multi-patch network architecture integrated with Context Module and additional ResBlocks in order to tackle deblurring problem. Compared with recently proposed networks, it generates images with better visual effect as well as higher image quality index. Besides, our model significantly reduces the test time. We evaluate the proposed network structure on public GoPro dataset, a large-scale image dataset with complex synthetic blur. The experiments on the benchmark dataset prove that our effective method outperforms other state-of-the-art blind deblurring algorithms both qualitatively and quantitatively, which demonstrates the effectiveness of Context Module in the task of single image blur removal.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

References

  1. Hyun Kim T, Ahn B, Mu Lee K (2013) Dynamic scene deblurring. In: ICCV, pp 3160–3167

  2. Sun J, Cao W, Xu Z, Ponce J (2015) Learning a convolutional neural network for non-uniform motion blur removal. In: CVPR, pp 769–777

  3. Nah S, Hyun Kim T, Mu Lee K (2017) Deep multi-scale convolutional neural network for dynamic scene deblurring. In: Proceedings of IEEE conference on computer vision and pattern recognition, pp 257–265

  4. Tao X, Gao H, Shen X, Wang J, Jia J (2018) Scale-recurrent network for deep image deblurring. In: CVPR, pp 8174–8182

  5. Mei J, Wu Z, Chen X et al (2019) DeepDeblur: text image recovery from blur to sharp. Multim Tools Appl 78(13):18869–18885

    Article  Google Scholar 

  6. Shen Z, Lai WS, Xu T, Kautz J, Yang MH (2018) Deep semantic face deblurring, pp 8260–8269. https://doi.org/10.1109/cvpr.2018.00862

  7. Ren W, Zhang J, Ma L, Pan J, Cao X, Zuo W, LiuW, Yang M-S (2018) Deep non-blind deconvolution via generalized low-rank approximation. In: Advances in neural information processing systems, pp 295–305

  8. Xiangyu Xu, Jinshan Pan, Yu-Jin Zhang, and Ming-HsuanYang. Motion blur kernel estimation via deep learning. IEEE Transactions on Image Processing, 27(1):194–205, 2018

  9. Li L, Pan J, Lai W-S, Gao X, Sang N, Yang M-H (2018) Learning a discriminative prior for blind image deblurring. In: The IEEE conference on computer vision and pattern recognition (CVPR), June 2018

  10. Chang C-F, Wu J-L (2014) A new single image deblurring algorithm using hyper laplacian priors. In: ICS, pp 1015–1022

  11. Xu L, Zheng S, Jia J (2013) Unnatural l0 sparse representation for natural image deblurring. In: CVPR

  12. Pan J, Sun D, Pfister H, Yang M-H (2016) Blind image deblurring using dark channel prior. In: CVPR

  13. Cao Y, Wang S, Guo Z et al (2017) Synchronization of memristive neural networks with leakage delay and parameters mismatch via event-triggered control. Neural Netw 2019:119

    Google Scholar 

  14. Wang Shengbo, Cao Yuting, Huang Tingwen, Chen Yiran, Wen Shiping (2020) Event-triggered distributed control for synchronization of multiple memristive neural networks under cyber-physical attacks. Inf Sci 518:361–375

    Article  MathSciNet  Google Scholar 

  15. Xu L, Ren JSJ, Liu C, Jia J (2014) Deep convolutional neural network for image deconvolution. In: Proceedings of advance neural information processing system, pp 1790–1798

  16. Gong D, Yang J, Liu L, Zhang Y, Reid ID, Shen C, Van Den Hengel A, Shi Q (2017) From motion blur to motion flow: a deep learning solution for removing heterogeneous motion blur. In: CVPR, 2017

  17. Schuler CJ, Hirsch M, Harmeling S, Schölkopf B (2016) Learning to deblur. IEEE Trans Pattern Anal Mach Intell 38(7):1439–1451

    Article  Google Scholar 

  18. Chakrabarti A (2016) A neural approach to blind motion deblurring. In: ECCV. Springer, pp 221–235

  19. Gao H, Tao X, Shen X, Jia J (2019) Dynamic scene deblurring with parameter selective sharing and nested skip connections. CVPR 2019, pp 3848–3856

  20. Ramakrishnan S, Pachori S, Gangopadhyay A, Raman S (2017) Deep generative filter for motion deblurring. In: 2017 IEEE international conference on computer vision workshops (ICCVW), pp 2993–3000

  21. Isola P, Zhu J-Y, Zhou T (2016) Image-to-image translation with conditional adversarial networks

  22. Kupyn O, Budzan V, Mykhailych M, Mishkin D, Matas J (2018) Deblurgan: blind motion deblurring using conditional adversarial networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 8183–8192

  23. Arjovsky M, Chintala S, Bottou L (2017) Wasserstein gan. arXiv:1701.07875

  24. Gulrajani I, Ahmed F, Arjovsky M, Dumoulin V, Courville AC (2017) Improved training of wasserstein gans. In: Advances in neural information processing systems, pp 5767–5777

  25. Kupyn O, Martyniuk T, Wu J, Wang Z (2019) DeblurGAN-v2: deblurring (orders-of-magnitude) faster and better. CoRR arXiv:1908.03826

  26. He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: CVPR, pp 770–778

  27. Lazebnik S, Schmid C, Ponce J (2006) Beyond bags of features: spatial pyramid matching for recognizing natural scene categories. In: Proceedings of IEEE computer society conference on computer vision and pattern recognition. IEEE, pp 2169–2178

  28. Zhang Hongguang, Dai Yuchao, Li Hongdong (2019) PiotrKoniusz. Deep stacked hierarchical multi-patch network for image deblurring with, IEEE Comput Vis Pattern Recognit (CVPR)

    Google Scholar 

  29. Chen LC, Papandreou G, Kokkinos I et al (2018) DeepLab: semantic Image segmentation with deep convolutional nets, atrous convolution, and fully connected CRFs. IEEE Trans Pattern Anal Mach Intell 40(4):834

    Article  Google Scholar 

  30. He K, Zhang X, Ren S et al (2014) Spatial pyramid pooling in deep convolutional networks for visual recognition. IEEE Trans Pattern Anal Mach Intel 37(9):1904–1916

    Article  Google Scholar 

  31. Noroozi M, Chandramouli P, Favaro P (2017) Motion deblurring in the wild. In: GCPR, 2017

  32. Zhou S, Zhang J, Zuo W, Xie H, Pan J, Ren JS (2019) DAVANet: stereo deblurring with view aggregation. In: CVPR 2019, pp 10996–11005

  33. Johnson J, Alahi A, Fei-Fei L (2016) Perceptual losses for real-time style transfer and super-resolution. Eur Conf Comput Vis 1(4):5

    Google Scholar 

  34. Zhao H, Gallo O, Frosio I et al (2017) Loss functions for image restoration with neural networks. IEEE Trans Comput Imaging 3(1):47–57

    Article  Google Scholar 

  35. Kohler R, Hirsch M, Mohler B, Scholkopf B, Harmeling S (2012) Recording and playback of camera shake: benchmarking blind deconvolution with a real-world database. In: ECCV. Springer, pp 27–40

  36. PyTorch. http://pytorch.org

  37. Kingma DP, Ba J (2014) Adam: a method for stochastic optimization. Computer Science

  38. Glorot X, Bengio Y (2010) Understanding the difficulty of training deep feed forward neural networks. Aistats 9:249–256

    Google Scholar 

  39. Zhang J, Pan J, Ren J, Song Y, Bao L, Lau RWH, Yang M-H (2018) Dynamic scene deblurring using spatially variant recurrent neural networks. In: Proceedings of IEEE conference on computer vision pattern recognition, pp 2521–2529

Download references

Author information

Authors and Affiliations

  1. College of Information Science and Engineering, Hunan Normal University, Changsha, 410081, China

    Kang Tang, Dahong Xu, Hong Liu & Zhixin Zeng

Authors
  1. Kang Tang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Dahong Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Hong Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Zhixin Zeng
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Kang Tang.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Tang, K., Xu, D., Liu, H. et al. Context Module Based Multi-patch Hierarchical Network for Motion Deblurring. Neural Process Lett 53, 211–226 (2021). https://doi.org/10.1007/s11063-020-10370-0

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11063-020-10370-0

Keywords

Search

Navigation