Skip to main content
SpringerLink
Log in

Stack-based Scale-recurrent Network for Face Image Deblurring

  • Published:
Neural Processing Letters Aims and scope Submit manuscript

Abstract

In recent years, the image deblurring task has attracted more and more researchers’ attention. Many researchers are devoted to eliminating motion blur by using a “coarse-to-fine” architecture. This architecture can effectively eliminate motion blur caused by a simple relative displacement. But when using the architecture directly for the face image deblurring task, there would exist some problems. For example, complex network structures make the model difficult to be trained, and a large number of parameters need to be calculated would result in expensive runtime. Since details of images can’t be completely restored, the quality of images will deteriorate, and deblurring visual effect is poor, so in order to solve these issues, combining the “coarse-to-fine” architecture with the “stacked” architecture, we propose a new network architecture-“Stack-based Scale-recurrent Network” for the face image deblurring task. The ConvLSTM network is employed to build a model. We achieves the purpose of improving the visual clarity and the quality of restored face images. And we use the improved encoder-decoder to enhance network performance. Compared with the other deblurring methods, our method can restore clear face images with higher-quality and clearer vision.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

References

  1. Wang Z, Yao Z, Wang Q (2017) Improved scheme of estimating motion blur parameters for image restoration[J]. Digit Sign Process 65:11–18

    Article  MathSciNet  Google Scholar 

  2. Hirsch M, Schuler C J, Harmeling S et al (2012) Fast removal of non-uniform camera shake[C]. In: IEEE International conference on computer vision. IEEE

  3. Harmeling S, Hirsch M, Schlkopf B (2010) Space-variant single-image blind deconvolution for removing camera shake[C]. Adv Neural Inf Process Syst 23:829–837

    Google Scholar 

  4. Perrone D, Favaro P (2016) A logarithmic image prior for blind deconvolution [J]. Int J Comput Vis 117(2):159–172

    Article  MathSciNet  Google Scholar 

  5. Jiang X, Yao H, Zhao S (2017) Text image deblurring via two-tone prior [J]. Neurocomputing 242:1–14

    Article  Google Scholar 

  6. Xu Z, Chen H, Li Z (2021) Fast blind deconvolution using a deeper sparse patch-wise maximum gradient prior[J]. Signal Process Image Commun 90:1114–1122

    Google Scholar 

  7. Pan J, Sun D, Pfifister H et al (2016) Blind image deblurring using dark channel prior[C]. In: IEEE Conference on computer vision and pattern recognition. IEEE, pp. 1628–1636

  8. Yi O (2019) Total variation constraint GAN for dynamic scene deblurring[J]. Image Vis Comput 88:113–119

    Article  Google Scholar 

  9. Kja B, Ying SC, Ql C et al (2021) Image restoration using overlapping group sparsity on hyper-Laplacian prior of image gradient[J]. Neurocomputing 420:57–69

    Article  Google Scholar 

  10. Zhou K, Zhuang P, Xiong J et al (2020) Blind image deblurring with joint extreme channels and L0-regularized intensity and gradient priors[C]. In: IEEE International conference on image processing (ICIP). IEEE

  11. Ledig C, Theis L, Huszar F, Caballero J, Cunningham A, Acosta A, Aitken A, Tejani A, Totz J, Wang Z, Shi W (2017) Photo-realistic single image super–resolution using a generative adversarial network. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), IEEE, pp 105–114

  12. Yda B, Zd C, Shuai YC et al (2020) Kernel-attended residual network for single image super-resolution - ScienceDirect[J]. Knowledge-Based Syst 213:52–60

    Google Scholar 

  13. Yu J, Tan M, Zhang H et al (2019) Hierarchical deep click feature prediction for fine-grained image recognition[J]. IEEE Trans Pattern Anal Mach Intell 99:1

    Google Scholar 

  14. Isola P, Zhu J Y, Zhou T et al (2016) Image-to-image translation with conditional adversarial networks[C]. In: IEEE Conference on computer vision and pattern recognition. IEEE

  15. Yu J, Kuang Z, Zhang B et al (2018) Leveraging content sensitiveness and user trustworthiness to recommend fine-grained privacy settings for social image sharing[J]. IEEE Trans Inf Forensics Secur 13:1317–1332

    Article  Google Scholar 

  16. Li Y, Huang JB, Ahuja N et al (2016) Deep joint image filtering[J]. In: European conference on computer vision. pp. 771–779

  17. Khan AT, Cao X, Li S et al (2021) Quantum beetle antennae search: a novel technique for the constrained portfolio optimization problem[J]. Sci China Inf Sci 64(5):152204

  18. Atk A, Shuai LA, Xc B (2021) Control framework for cooperative robots in smart home using bio-inspired neural network - ScienceDirect[J]. Measurement 167:108253

    Article  Google Scholar 

  19. Gampala V, Kumar M, Sushama C, Sehar E, Raj FI (2020) Deep learning based image processing approaches for image deblurring. Mater Today Proc 2020:601–609

  20. Zhang J, Pan J, Ren J et al (2018) Dynamic scene deblurring using spatially variant recurrent neural networks[C]. In: IEEE/CVF Conference on computer vision and pattern recognition. IEEE

  21. Huang L, Xia Y (2019) Joint blur Kernel estimation and CNN for blind image restoration[J]. Neurocomputing 396:562–570

    Google Scholar 

  22. Singhal J, DeblurRL Narang P (2021) Image deblurring with deep reinforcement learning[M]. Electronic Industry Press, Beijing

    Google Scholar 

  23. Wang M, Hou S, Li H et al (2019) Generative image deblurring based on multi-scaled residual adversary network driven by composed prior-posterior loss[J]. J Vis Commun Image Represent 65:1621–1629

    Google Scholar 

  24. Xu L, Ren JS, Liu C et al (2014) Deep convolutional neural network for image deconvolution[C]. NIPS 27:1790–1798

    Google Scholar 

  25. Gong D, Yang J, Liu L et al (2017) From motion blur to motion flflow: a deep learning solution for removing heterogeneous motion blur[C]. In: CVPR

  26. Noroozi M, Chandramouli P, Favaro P (2017) Motion deblurring in the wild[C]. In: German conference on pattern recognition

  27. Sun J, Cao W, Xu Z et al (2015) Learning a convolutional neural network for non-uniform motion blur removal[J]. In: IEEE Conference on computer vision and pattern recognition pp. 235–243

  28. Nah S, Kim TH, Lee KM (2017) Deep multi-scale convolutional neural network for dynamic scene deblurring. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), IEEE, pp 257–265

  29. Chakrabarti A (2016) A neural approach to blind motion deblurring[C]. In: European conference on computer vision. Springer, Cham, pp 221–235

  30. Dong G, Jie Y, Liu L et al (2017) From Motion blur to motion flow: a deep learning solution for removing heterogeneous motion blur[C]. In: IEEE Conference on computer vision and pattern recognition. IEEE

  31. Seungjun Nah, Tae Hyun Kim, Kyoung Mu Lee (2017) Deep multi-scale convolutional neural network for dynamic scene deblurring[C]. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 257–265

  32. Mao XJ, Shen C, Yang YB (2016) Image restoration using very deep convolutional encoder-decoder networks with symmetric skip connections[J]. In: Conference on neural information processing systems pp. 2802–2810

  33. Hongguang Zhang, Yuchao Dai, Hongdong Li et al (2019) Deep stacked hierarchical multi-patch network for image deblurring[C]. In: CVPR, pp.5979-5986

  34. Kupyn O, Budzan V, Mykhailych M et al (2018) DeblurGAN: Blind motion deblurring using conditional adversarial networks[C]. In: IEEE/CVF Conference on computer vision and pattern recognition. IEEE

  35. Yu J, Zhu C, Zhang J et al (2019) Spatial pyramid-enhanced NetVLAD with weighted triplet loss for place recognition[J]. IEEE Trans Neural Netw Learn Syst 31:1–14

    Google Scholar 

  36. Eigen D, Fergus R (2014) Predicting depth, surface normals and semantic labels with a common multi-scale convolutional architecture[C]. In: IEEE. IEEE

  37. Mathieu MF, Couprie C, Cun Y (2018) Deep multi-scale video prediction. US20180137389[P]

  38. Dosovitskiy A, Fischer P, Ilg E et al (2015) Flownet: Learning optical flflow with convolutional networks. In: CVPR, pp. 2758–2766

  39. Khan AH, Li S, Luo X (2020) Obstacle avoidance and tracking control of redundant robotic manipulator: an RNN-based metaheuristic approach[J]. IEEE Trans Ind Inform 16(7):4670–4680

    Article  Google Scholar 

  40. Khan AH, Li S, Cao X (2021) Tracking control of redundant manipulator under active remote center-of-motion constraints: an RNN-based metaheuristic approach[J]. Sci China Inf Sci 64(3):1–18

    Article  MathSciNet  Google Scholar 

  41. Ahk A, Shuai LB, Dc C et al (2020) Tracking control of redundant mobile manipulator: an RNN based metaheuristic approach[J]. Neurocomputing 400:272–284

    Article  Google Scholar 

  42. Tao X, Gao H, Shen X, Wang J, Jia J (2018) Scale-recurrent network for deep image deblurring[C]. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 8174–8182

  43. Liu Z, Yeh R, Tang X et al (2017) Video frame synthesis using deep voxel flow[J]. In: IEEE

  44. Su S, Delbracio M, Wang J et al. (2016) Deep video deblurring [J]

  45. Tao X, Gao H, Liao R et al (2017) Detail-revealing deep video super-resolution. In: IEEE Computer society

  46. Xu N, Price B, Cohen S et al (2017) Deep Image Matting[C]. In: IEEE Conference on computer vision and pattern recognition. IEEE Computer society, pp. 311–320

  47. He K, Zhang X, Ren S et al (2016) Deep Residual Learning for Image Recognition[C]. In: 2016 IEEE Conference on computer vision and pattern recognition (CVPR). IEEE, 2016

  48. Kingma D, Adam Ba J (2014) A method for stochastic optimization[J]. Computer. Science

Download references

Acknowledgements

This work was partly supported in part by the National Natural Science Foundation of China (No. 61871464 and 61836002), the Fujian Provincial Natural Science Foundation of China (No. 2018J01573 and 2019J01854), the Open Project Program of State Key Laboratory of Virtual Reality Technology and Systems, Beihang University (No. VRLAB2021C01), Distinguished Young Scientific Research Talents Plan in Universities of Fujian Province, the Program for New Century Excellent Talents in University of Fujian Province and Xiamen Youth Innovation Fund Project (No. 3502Z20206072).

Author information

Authors and Affiliations

  1. Xiamen University of Technology, Xiamen, China

    Yanqiu Wu, Chaoqun Hong, Xuebai Zhang & Yifan He

Authors
  1. Yanqiu Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Chaoqun Hong
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Xuebai Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yifan He
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Chaoqun Hong.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wu, Y., Hong, C., Zhang, X. et al. Stack-based Scale-recurrent Network for Face Image Deblurring. Neural Process Lett 53, 4419–4436 (2021). https://doi.org/10.1007/s11063-021-10604-9

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11063-021-10604-9

Keywords

Search

Navigation