Skip to main content
SpringerLink
Log in

Blind face images deblurring with enhancement

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Face images deblurring has achieved advanced development; however, existing methods involve high computational cost problems. Furthermore, the recovered face images by current methods have the problems of over-smooth textures, ringing artifacts, and poor details. We consider the problem of face images deblurring as a semantic generation task. In this paper, we propose a generative adversarial network (GAN), which includes a perception-inspired blurry removal generator and a discriminator. The proposed generator reconstructs the latent deblurred image by a U-net based network that contains an enhancement module. Face images are highly structured, and thus can be served as a class-specific prior. Considering this, we propose a perceptual loss function to regularize the recovery of face images, which introduces more clear details and reduces the effects of artifacts. The proposed method has a robust capability of generating realistic face images with pleasant visual effects. Extensive experiments on both synthetic and real-world face images demonstrate that the proposed method is comparable with state-of-the-art methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

References

  1. Alom MZ, Hasan M, Yakopcic C, Taha TM, Asari VK (2018) Improved inception-residual convolutional neural network for object recognition

  2. Anwar S, Phuoc Huynh C, Porikli F (2015) Class-specific image deblurring

  3. Ayan C (2016) a neural approach to blind motion deblurring

  4. Boracchi G, Foi A (2012) Modeling the performance of image restoration from motion blur. IEEE Trans Image Proc 21:3502–3517

    Article  MathSciNet  Google Scholar 

  5. Cho S, Lee S (2009) Fast motion deblurring

  6. Chrysos GG, Zafeiriou S (2017) Deep face deblurring

  7. Cronje J (2015) Deep convolutional neural networks for dense non-uniform motion deblurring

  8. Fergus R, Singh B, Hertzmann A, Roweis ST, Freeman WT (2006) Removing camera shake from a single photograph. ACM transactions on graphics 25:787–794

    Article  Google Scholar 

  9. Goodfellow I, Pouget-Abadie J, Mirza M, Xu B, Warde-Farley D, Ozair S, Bengio Y, et al (2014) Generative adversarial nets, Proc

  10. Gulrajani I, Ahmed F, Arjovsky M, Dumoulin V, Courville AC (2017) Improved training of wasserstein gans

  11. Hacohen Y, Shechtman E, Lischinski D (2013) Deblurring by example using dense correspondence

  12. He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition

  13. Hradiš M., Kotera J, Zemcík P, Šroubek F (2015) Convolutional neural networks for direct text deblurring. Proc British machine vis Conf 10:2

    Google Scholar 

  14. Isola P, Zhu JY, Zhou T, Efros AA (2017) Image-to-image translation with conditional adversarial networks

  15. Jin M, Hirsch M, Favaro P, fast Learning face deblurring, wide Proc. IEEE Conf. Comput. Vis. (2018) Patt Recog. workshops

  16. Johnson J, Alahi A, Fei-Fei L (2016) Perceptual losses for real-time style transfer and super-resolution

  17. Joshi N, Szeliski R, Kriegman DJ (2008) Psf estimation using sharp edge prediction

  18. Kingma DP, Ba J (2014). arXiv:1412.6980

  19. Krishnan D, Tay T, Fergus R (2011) Blind deconvolution using a normalized sparsity measure

  20. Kupyn O, Budzan V, Mykhailych M, Mishkin D, Matas J (2018) Deblurgan: Blind motion deblurring using conditional adversarial networks

  21. Kupyn O, Martyniuk T, Wu J, Wang Z (2019) Deblurgan-v2: Deblurring (orders-of-magnitude) faster and better

  22. Le V, Brandt J, Lin Z, Bourdev L, Huang TS (2012) Interactive facial feature localization

  23. Levin A, Weiss Y, Durand F, Freeman WT (2009) Understanding and evaluating blind deconvolution algorithms

  24. Li C, Anwar S, Porikli F (2019) Underwater scene prior inspired deep underwater image and video enhancement Pattern Recognition

  25. Li C, Cong R, Hou J, Zhang S, Qian Y, Kwong S (2019). arXiv:1901.05495

  26. Li C, Cong R, Hou J, Zhang S, Qian Y, Kwong S (2019). arXiv:1906.08462

  27. Li C, Guo J, Guo C (2018) Emerging from water: Underwater image color correction based on weakly supervised color transfer. IEEE Signal processing letters 25(3):323–327

    Article  Google Scholar 

  28. Li C, Guo C, Guo J, Han P, Fu H, Cong R (2019) PDR-Net Perception-Inspired Single Image Dehazing Network with Refinement

  29. Li X, Liu M, Ye Y, Zuo W, Lin L, Yang R (2018) Learning warped guidance for blind face restoration

  30. Liu Z, Luo P, Wang X, Tang X (2015) Deep learning face attributes in the wild

  31. Maas AL, Hannun AY, Ng AY (2013) Rectifier nonlinearities improve neural network acoustic models. Proc Int Conf Machine Learn 30:3

    Google Scholar 

  32. Mao X, Shen C, Yang YB (2016) Image restoration using very deep convolutional encoder-decoder networks with symmetric skip connections

  33. Nah S, Kim TH, Lee KM (2017) Deep multi-scale convolutional neural network for dynamic scene deblurring

  34. Nishiyama M, Hadid A, Takeshima H, Shotton J, Kozakaya T, Yamaguchi O (2011) Facial deblur inference using subspace analysis for recognition of blurred faces. IEEE Trans patt analy machine intel 33:838–845

    Article  Google Scholar 

  35. Pan J, Hu Z, Su Z, Yang MH (2014) Deblurring face images with exemplars

  36. Pan J, Sun D, Pfister H, Yang MH (2016) Blind image deblurring using dark channel prior

  37. Ren W, Cao X, Pan J, Guo X, Zuo W, Yang MH (2016) Image deblurring via enhanced low-rank prior. IEEE Trans. Image Proc. 25:3426–3437

    Article  MathSciNet  Google Scholar 

  38. Ronneberger O, Fischer P, Brox T (2015) U-net: Convolutional networks for biomedical image segmentation

  39. Schuler CJ, Hirsch M, Harmeling S, Schölkopf B (2016) Learning to deblur. IEEE Trans. patt. analy. machine intel. 38:1439–1451

    Article  Google Scholar 

  40. Shi W, Caballero J, Huszr F, Totz J, Aitken AP, Bishop R, et al (2016) Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network

  41. Simonyan K (2014). arXiv:1409.1556

  42. Su S, Delbracio M, Wang J, Sapiro G, Heidrich W, Wang O (2017) Deep video deblurring for hand-held cameras. In: Proc IEEE Int Conf Comput Vis, p 6

  43. Sun L, Cho S, Wang J, Hays J (2013) Edge-based blur kernel estimation using patch priors

  44. Ulyanov D, Vedaldi A, Lempitsky V (2016). arXiv:1607.08022

  45. Wang Z, Bovik AC, Sheikh HR, Simoncelli EP (2004) Image quality assessment: from error visibility to structural similarity. IEEE Trans Image Process 13:600–612

    Article  Google Scholar 

  46. Wang X, Yu K, Wu S, Gu J, Liu Y, Dong C, et al (2018) Esrgan: Enhanced super-resolution generative adversarial networks

  47. Wen F, Ying R, Liu P, Truong TK (2019) Blind Image Deblurring Using Patch-Wise Minimal Pixels Regularization. arXiv:1906.06642

  48. Wieschollek P, Hirsch M, Schölkopf B., Lensch HP (2017) Learning blind motion deblurring

  49. Xu L, Jia J (2010) Two-phase kernel estimation for robust motion deblurring

  50. Xu L, Ren JS, Liu C, Jia J (2014) Deep convolutional neural network for image deconvolution

  51. Xu L, Zheng S, Jia J (2013) Unnatural l0 sparse representation for natural image deblurring

  52. Yan R, Shao L (2016) Blind image blur estimation via deep learning. IEEE Trans. Image Proc. 25:1910–1921

    MathSciNet  MATH  Google Scholar 

  53. Zhang H, Yang J, Zhang Y, Huang TS (2011) Sparse representation based blind image deblurring

  54. Zhong L, Cho S, Metaxas D, Paris S, Wang J (2013) Handling noise in single image deblurring using directional filters

Download references

Acknowledgements

This document is the results of the research project funded by the National Science Foundation of China Grant No.61771334, the ChunHui project, Ministry of education, China No.Z2016105.

Author information

Authors and Affiliations

  1. School of Electrical and Information Engineering, Tianjin University, Weijin Road 92, 300072, Tianjin, China

    Qing Qi, Jichang Guo & Lijun Xiao

  2. School of Physics and Electronic Information Engineering, Qinghai Nationalities University, Bayi Middle Road 3, 810007, Xining, China

    Qing Qi

  3. Department of Computer Science, City University of Hong Kong, Tat Chee Avenue, Kowloon, 999077, Hong Kong, China

    Chongyi Li

Authors
  1. Qing Qi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jichang Guo
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Chongyi Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Lijun Xiao
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jichang Guo.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Qi, Q., Guo, J., Li, C. et al. Blind face images deblurring with enhancement. Multimed Tools Appl 80, 2975–2995 (2021). https://doi.org/10.1007/s11042-020-09460-x

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-020-09460-x

Keywords

Search

Navigation