Skip to main content
SpringerLink
Log in

Mixed Gaussian-Impulse noise robust face hallucination via noise suppressed low-and-high resolution space-based neighbor representation

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

An intelligent surveillance system poses a lot of challenges in the processing of captured noisy low-resolution (LR) images. To defeat such challenges, face super-resolution (SR) also called face hallucination techniques are getting prominence in recent years. Although, the present SR models are not good enough to handle the complicated noise e.g., mixed Gaussian-Impulse (MGI) noise, often present in the captured LR images. Therefore, a new MGI noise-robust face hallucination algorithm using noise suppressed low-and-high resolution space-based neighbor representation (NSLHNR) is proposed in this paper. The proposed algorithm first suppresses the effect of outliers from the SR process by overlooking them from the reconstruction weight calculation process. It assists in controlling the square reconstruction error. Further, it also accomplishes the HR space-based neighbor representation to counterbalance the losses caused due to high-density MGI noise in a relationship of input and training LR images. These additions make the proposed algorithm capable to preserve sharp edges, texture, and the individual characteristics of the input face in the output. The performance measured through the experiments performed on the benchmark datasets and surveillance images shows the better reconstruction capability of the proposed algorithm over the compared state-of-the-art models.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

Notes

  1. Given optimization steps are also followed to simplify function in (10)

  2. Source code is available on email request to ershyamrajput@gmail.com

  3. The position-patch prior [14] is employed with the NE method to make the comparison fair.

References

  1. Baker S, Kanade T (2000) Hallucinating faces. In: Proceedings Fourth IEEE International Conference on Automatic Face and Gesture Recognition(Cat. No. PR00580), pp. 83–88

  2. Baker S, Kanade T (2002) Limits on super-resolution and how to break them. IEEE Transactions on Pattern Analysis and Machine Intelligence 24(9):1167–1183

    Article  Google Scholar 

  3. Cao Q, Lin L, Shi Y, Liang X, Li G (2017) Attention-aware face hallucination via deep reinforcement learning 2017 IEEE Conference on computer vision and pattern recognition (CVPR), pp. 1656–1664

  4. Chakrabarti A, Rajagopalan AN, Chellappa R (2007) Super-resolution of face images using kernel pca-based prior. IEEE Transactions on Multimedia 9 (4):888–892

    Article  Google Scholar 

  5. Chang H, Yeung DY, Xiong Y (2004) Super-resolution through neighbor embedding. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), vol. 1, pp. 1–8

  6. Chen CLP, Liu L, Chen L, Tang YY, Zhou Y (2015) Weighted couple sparse representation with classified regularization for impulse noise removal. IEEE Trans Image Process 24(11):4014–4026

    Article  MathSciNet  Google Scholar 

  7. Chen Y, Tai Y, Liu X, Shen C, Yang J (2018) Fsrnet: End-to-end learning face super-resolution with facial priors 2018 IEEE/CVF Conference on computer vision and pattern recognition, pp. 2492–2501

  8. Dong C, Loy CC, He K, Tang X (2016) Image super-resolution using deep convolutional networks. IEEE Transactions on Pattern Analysis and Machine Intelligence 38(2):295–307

    Article  Google Scholar 

  9. Huang H, He H, Fan X, Zhang J (2010) Super-resolution of human face image using canonical correlation analysis. Pattern Recognition 43 (7):2532–2543

    Article  Google Scholar 

  10. Huang H, He R, Sun Z, Tan T (2017) Wavelet-srnet: a wavelet-based cnn for multi-scale face super resolution. In: 2017 IEEE International conference on computer vision (ICCV), pp. 1698–1706

  11. Huang H, Wu N (2011) Fast facial image super-resolution via local linear transformations for resource-limited applications. IEEE Transactions on Circuits and Systems for Video Technology 21(10):1363–1377

    Article  Google Scholar 

  12. Hwang H, Haddad RA (1995) Adaptive median filters: new algorithms and results. IEEE Trans Image Process 4(4):499–502

    Article  Google Scholar 

  13. Jia K, Gong S (2008) Generalized face super-resolution. IEEE Trans Image Process 17(6):873–886

    Article  MathSciNet  Google Scholar 

  14. Jiang J, Chen C, Huang K, Cai Z, Hu R (2016) Noise robust position-patch based face super-resolution via tikhonov regularized neighbor representation. Information Sciences 367–368:354 –372

    Article  Google Scholar 

  15. Jiang J, Hu R, Wang Z, Han Z (2014) Noise robust face hallucination via locality-constrained representation. IEEE Transactions on Multimedia 16 (5):1268–1281

    Article  Google Scholar 

  16. Jiang J, Ma J, Chen C, Jiang X, Wang Z (2016) Noise robust face image super-resolution through smooth sparse representation. IEEE Transactions on Cybernetics 47(11):3991–4002

    Article  Google Scholar 

  17. Kim J, Lee JK, Lee KM (2016) Accurate image super-resolution using very deep convolutional networks. In: 2016 IEEE Conference on computer vision and pattern recognition (CVPR), pp. 1646–1654

  18. Liu C, Shum HY, Zhang CS (2001) A two-step approach to hallucinating faces: global parametric model and local nonparametric model. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR 2001), vol. 1, pp. 192–198

  19. Liu L, Chen CLP, Li S, Tang YY, Chen L (2018) Robust face hallucination via locality-constrained bi-layer representation. IEEE Transactions on Cybernetics 48(4):1189–1201

    Article  Google Scholar 

  20. Ma X, Zhang J, Qi C (2010) Hallucinating face by position-patch. Pattern Recognition 43(6):2224–2236

    Article  Google Scholar 

  21. Park JS, Lee SW (2008) An example-based face hallucination method for single-frame, low-resolution facial images. IEEE Trans Image Process 17 (10):1806–1816

    Article  MathSciNet  Google Scholar 

  22. Rajput SS, Arya K, Singh V (2018) Robust face super-resolution via iterative sparsity and locality-constrained representation. Information Sciences 463–464:227–244

    Article  Google Scholar 

  23. Rajput SS, Arya KV (2019) Noise robust face hallucination via outlier regularized least square and neighbor representation. IEEE Transactions on Biometrics. Behavior, and Identity Science 1(4):252–263

    Google Scholar 

  24. Rajput SS, Arya K (2019) A robust facial image super-resolution model via mirror-patch based neighbor representation. Multimedia Tools and Applications 78:1324–1338

    Article  Google Scholar 

  25. Rajput SS, Arya K (2020) A robust face super-resolution algorithm and its application in low-resolution face recognition system. Multimedia Tools and Applications 79(33):23909–23934

    Article  Google Scholar 

  26. Rajput SS, Arya KV, Singh V, Bohat VK (2018) Face hallucination techniques: a survey. In: 2018 Conference on information and communication technology (CICT), pp. 1–6

  27. Rajput SS, Bohat VK, Arya K (2019) Grey wolf optimization algorithm for facial image super-resolution. Appl Intell 49:1324–1338

    Article  Google Scholar 

  28. Rajput SS, Singh A, Arya K, Jiang J (2018) Noise robust face hallucination algorithm using local content prior based error shrunk nearest neighbors representation. Signal Processing 147:233–246

    Article  Google Scholar 

  29. Roweis ST, Saul LK (2000) Nonlinear dimensionality reduction by locally linear embedding. Science 290(5500):2323–2326

    Article  Google Scholar 

  30. Rowley HA, Baluja S, Kanade T (1998) Neural network-based face detection. IEEE Transactions on Pattern Analysis and Machine Intelligence 20(1):23–38

    Article  Google Scholar 

  31. Thomaz CE, Giraldi GA (2010) A new ranking method for principal components analysis and its application to face image analysis. Image and Vision Computing 28(6):902–913

    Article  Google Scholar 

  32. Wang J, Yang J, Yu K, Lv F, Huang T, Gong Y (2010) Locality-constrained linear coding for image classification 2010 IEEE Computer society conference on computer vision and pattern recognition, pp. 3360–3367

  33. Wang N, Tao D, Gao X, Li X, Li J (2014) A comprehensive survey to face hallucination. Int J Comput Vis 106(1):9–30

    Article  Google Scholar 

  34. Wang X, Tang X (2005) Hallucinating face by eigentransformation. IEEE Transactions on Systems, Man, and Cybernetics. Part C (Applications and Reviews) 35(3):425–434

    Google Scholar 

  35. Wang Z, Hu R, Wang S, Jiang J (2014) Face hallucination via weighted adaptive sparse regularization. IEEE Transactions on Circuits and Systems for Video Technology 24(5):802–813

    Article  Google Scholar 

  36. Wang Z, Liu D, Yang J, Han W, Huang T (2015) Deep networks for image super-resolution with sparse prior. In: 2015 IEEE International conference on computer vision (ICCV), pp. 370–378

  37. Yang J, Tang H, Ma Y, Huang T (2008) Face hallucination via sparse coding 2008 15Th IEEE international conference on image processing, pp. 1264–1267

  38. Yang J, Wright J, Huang TS, Ma Y (2010) Image super-resolution via sparse representation. IEEE Trans Image Process 19(11):2861–2873

    Article  MathSciNet  Google Scholar 

  39. Yu X, Porikli F (2018) Imagining the unimaginable faces by deconvolutional networks. IEEE Trans Image Process 27(6):2747–2761

    Article  MathSciNet  Google Scholar 

  40. Zhang W, Cham WK (2011) Hallucinating face in the dct domain. IEEE Trans Image Process 20(10):2769–2779

    Article  MathSciNet  Google Scholar 

  41. Zhang X, Peng S, Jiang J (2008) An adaptive learning method for face hallucination using locality preserving projections. In: 2008 8Th IEEE international conference on automatic face gesture recognition, pp. 1–8

  42. Zhu S, Liu S, Loy CC, Tang X Leibe B., Matas J., Sebe N., Welling M. (eds) (2016) Deep cascaded bi-network for face hallucination. Springer International Publishing, Cham

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science & Engineering, National Institute of Technology Patna, Patna-800005, Bihar, India

    Shyam Singh Rajput

Authors
  1. Shyam Singh Rajput
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Shyam Singh Rajput.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Rajput, S.S. Mixed Gaussian-Impulse noise robust face hallucination via noise suppressed low-and-high resolution space-based neighbor representation. Multimed Tools Appl 81, 15997–16019 (2022). https://doi.org/10.1007/s11042-022-12154-1

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-022-12154-1

Keywords

Search

Navigation