Skip to main content
SpringerLink
Log in

Parallel deep learning architecture with customized and learnable filters for low-resolution face recognition

  • Original Article
  • Published:
The Visual Computer Aims and scope Submit manuscript

Abstract

Face recognition in visual surveillance systems is important for various applications to identify individuals who are behaving defiantly at the time of an event or for investigation purposes. Despite the dramatic improvements in facial recognition technology in recent years, it is difficult to recognize faces from surveillance feeds due to the presence of multiple people of different scales and orientations. This paper solves the task of low-resolution face recognition by combining exemplary techniques for extracting distinct features. This research utilizes the attributes learned by customized and learnable filters and injected in the training process to better match them with human brain functionality. The Gabor transform aims to convolve a facial image using a range of Gabor filter coefficients at various scales and orientations, resulting in scale and rotation invariant features. The tailored architecture with residual stream aims to enhance functional representation and prevent the gradient of the prediction engine from affecting the backbone network functional map. Experimental analysis is performed on the SCface and TinyFace databases and is reported with an accuracy of 89.21% on the SCface database and 56.68% on the TinyFace database.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

References

  1. Hennings-Yeomans, P.H., Baker, S., Kumar, B.V.K.V.: Simultaneous super-resolution and feature extraction for recognition of low-resolution faces. In: 26th IEEE Conference Computer Vision and Pattern Recognition, CVPR. (2008). https://doi.org/10.1109/CVPR.2008.4587810

  2. Gunturk, B.K., Batur, A.U., Altunbasak, Y., Hayes, M.H., Mersereau, R.M.: Eigenface-domain super-resolution for face recognition. IEEE Trans. Image Process. 12, 597–606 (2003). https://doi.org/10.1109/TIP.2003.811513

    Article  Google Scholar 

  3. Fookes, C., Lin, F., Chandran, V., Sridharan, S.: Evaluation of image resolution and super-resolution on face recognition performance. J. Vis. Commun. Image Represent. 23, 75–93 (2012). https://doi.org/10.1016/j.jvcir.2011.06.004

    Article  Google Scholar 

  4. Gupta, S., Thakur, K., Kumar, M.: 2D-human face recognition using SIFT and SURF descriptors of face’s feature regions. Vis. Comput. 37, 447–456 (2021). https://doi.org/10.1007/s00371-020-01814-8

    Article  Google Scholar 

  5. Meedeniya, D.A., Ratnaweera, D.A.A.C.: Enhanced face recognition through variation of Principle component analysis (PCA). In ICIIS 2007 - 2nd International Conference on Industrial and Information Systems, pp. 347–351 (2007). https://doi.org/10.1109/ICIINFS.2007.4579200

  6. Wang, Z., Chang, S., Yang, Y., Liu, D., Huang, T.S.: Studying very low resolution recognition using deep networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4792–4800 (2016). https://doi.org/10.1109/CVPR.2016.518

  7. Zou, W.W.W., Yuen, P.C.: Very low resolution face recognition problem. IEEE Trans. Image Process. 21, 327–340 (2012). https://doi.org/10.1109/TIP.2011.2162423

    Article  MathSciNet  MATH  Google Scholar 

  8. Obara, K., Yoshimura, H., Nishiyama, M., Iwai, Y.: Low-resolution person recognition using image downsampling. In Proceedings 15th IAPR International Conference on Machine Vision Applications MVA, pp. 478–481 (2017). https://doi.org/10.23919/MVA.2017.7986904

  9. Peng, Y., Spreeuwers, L., Veldhuis, R.: Designing a low-resolution face recognition system for long-range surveillance. In Lecture Notes Informatics (LNI), Proc. - Ser. Gesellschaft fur Inform. P-260, (2016). https://doi.org/10.1109/BIOSIG.2016.7736917

  10. Hennings-Yeomans, P.H., Vijaya Kumar, B.V.K., Baker, S.: Robust low-resolution face identification and verification using high-resolution features. In Proceedings International Conference on Image Processing ICIP, pp. 33–36 (2009). https://doi.org/10.1109/ICIP.2009.5413920

  11. Yang, F., Yang, W., Gao, R., Liao, Q.: Discriminative multidimensional scaling for low-resolution face recognition. IEEE Signal Process. Lett. 25, 388–392 (2018). https://doi.org/10.1109/LSP.2017.2746658

    Article  Google Scholar 

  12. Zou, W., Yuen, P.C.: Very low resolution face recognition problem. In IEEE 4th International Conference on Biometrics: Theory, Applications and Systems BTAS (2010). https://doi.org/10.1109/BTAS.2010.5634490

  13. Jiang, J., Chen, C., Ma, J., Wang, Z., Wang, Z., Hu, R.: SRLSP: a face image super-resolution algorithm using smooth regression with local structure prior. IEEE Trans. Multimed. 19, 27–40 (2017). https://doi.org/10.1109/TMM.2016.2601020

    Article  Google Scholar 

  14. Mudunuri, S.P., Sanyal, S., Biswas, S.: GenLR-Net: Deep framework for very low resolution face and object recognition with generalization to unseen categories. In IEEE Conference on Computer Vision and Pattern Recognition, pp. 602–611 (2018). https://doi.org/10.1109/CVPRW.2018.00090

  15. Zhang, P., Ben, X., Jiang, W., Yan, R., Zhang, Y.: Coupled marginal discriminant mappings for low-resolution face recognition. Optik (Stuttg) 126, 4352–4357 (2015). https://doi.org/10.1016/j.ijleo.2015.08.138

    Article  Google Scholar 

  16. Maity, S., Abdel-Mottaleb, M., Asfour, S.S.: Multimodal low resolution face and frontal gait recognition from surveillance video. Electron (2021). https://doi.org/10.3390/electronics10091013

    Article  Google Scholar 

  17. Mishra, N.K., Dutta, M., Singh, S.K.: Multiscale parallel deep CNN (mpdCNN) architecture for the real low-resolution face recognition for surveillance. Image Vis. Comput. 115, 104290 (2021). https://doi.org/10.1016/J.IMAVIS.2021.104290

    Article  Google Scholar 

  18. Li, C., Huang, Y., Huang, W., Qin, F.: Learning features from covariance matrix of gabor wavelet for face recognition under adverse conditions. Pattern Recognit. (2021). https://doi.org/10.1016/j.patcog.2021.108085

    Article  Google Scholar 

  19. Rouhsedaghat, M., Wang, Y., Hu, S., You, S., Kuo, C.C.J.: Low-resolution face recognition in resource-constrained environments. Pattern Recognit. Lett. 149, 193–199 (2021). https://doi.org/10.1016/j.patrec.2021.05.009

    Article  Google Scholar 

  20. Zangeneh, E., Rahmati, M., Mohsenzadeh, Y.: Low resolution face recognition using a two-branch deep convolutional neural network architecture. Expert Syst. Appl. 139, 112854 (2020). https://doi.org/10.1016/j.eswa.2019.112854

    Article  Google Scholar 

  21. Cheng, Z., Zhu, X., Gong, S.: Low-resolution face recognition (including Subser). In Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics. 11363 LNCS, pp. 605–621 (2019). https://doi.org/10.1007/978-3-030-20893-6_38

  22. Xie, J., Ge, Y., Zhang, J., Huang, S., Chen, F., Wang, H.: Low-resolution assisted three-stream network for person re-identification. Vis. Comput. 38, 2515–2525 (2022). https://doi.org/10.1007/s00371-021-02127-0

    Article  Google Scholar 

  23. He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016). https://doi.org/10.1109/CVPR.2016.90

  24. Hubel, D.H., Wiesel, T.N.: Receptive fields, binocular interaction and functional architecture in the cat’s visual cortex. J. Physiol. (Lond.) 160, 106–154 (1962)

    Article  Google Scholar 

  25. Grgic, M., Delac, K., Grgic, S.: SCface - surveillance cameras face database. Multimed. Tools Appl. 51, 863–879 (2011). https://doi.org/10.1007/s11042-009-0417-2

    Article  Google Scholar 

  26. Hu, P., Ramanan, D.: Finding tiny faces. In Proceedings 30th IEEE Conference on Computer Vision and Pattern Recognition, pp. 1522–1530 (2017). https://doi.org/10.1109/CVPR.2017.166

  27. X. Yin, Y. Tai, Y. Huang, X. Liu, Fan: Feature adaptation network for surveillance face recognition and normalization. In: Proceedings of the Asian Conference on Computer Vision (ACCV) 2020)

  28. Lu, Z., Jiang, X., Kot, A.: Deep coupled ResNet for low-resolution face recognition. IEEE Signal Process. Lett. 25(4), 526–530 (2018). https://doi.org/10.1109/LSP.2018.2810121

    Article  Google Scholar 

  29. Y. Huang, P. Shen, Y. Tai, S. Li, X. Liu, J. Li, F. Huang, R. Ji, Improving face recognition from hard samples via distribution distillation loss. In: Proceedings of the 13th European Conference on Computer vision (ECCV), (Springer 2020), pp. 138–154.

  30. Song, Y., Tang, H., Meng, F., Wang, C., Wu, M., Shu, Z., Tong, G.: A transformer-based low-resolution face recognition method via on-and-offline knowledge distillation. Neurocomputing 509, 193–205 (2022). https://doi.org/10.1016/j.neucom.2022.08.058

    Article  Google Scholar 

  31. Gao, X., Sun, Y., Xiao, Y., Gu, Y., Chai, S., Chen, B.: Adaptive random down-sampling data augmentation and area attention pooling for low resolution face recognition. Expert Syst. Appl. 209, 118275 (2022). https://doi.org/10.1016/j.eswa.2022.118275

    Article  Google Scholar 

  32. Uzun-Per, M., Gökmen, M.: Face recognition with patch-based local Walsh transform. Signal Process. Image Commun. 61, 85–96 (2018). https://doi.org/10.1016/j.image.2017.11.003

    Article  Google Scholar 

  33. Mudunuri, S.P., Biswas, S.: Low resolution face recognition across variations in pose and illumination. IEEE Trans. Pattern Anal. Mach. Intell. 38(5), 1034–1040 (2015)

    Article  Google Scholar 

  34. Yang, F., Yang, W., Gao, R., Liao, Q.: Discriminative multidimensional scaling for low-resolution face recognition. IEEE Signal Process. Lett. 25(3), 388–392 (2017)

    Article  Google Scholar 

  35. Li, P., Prieto, L., Mery, D., Flynn, P.J.: On low-resolution face recognition in the wild: comparisons and new techniques. IEEE Trans. Inf. Forensics Secur. 14, 2000–2012 (2019). https://doi.org/10.1109/TIFS.2018.2890812

    Article  Google Scholar 

  36. Khalid, S.S., Awais, M., Feng, Z.H., Chan, C.H., Farooq, A., Akbari, A., Kittler, J.: Resolution invariant face recognition using a distillation approach. IEEE Trans. Biometr. Behav. Identity Sci. 2, 410–420 (2020). https://doi.org/10.1109/TBIOM.2020.3007356

    Article  Google Scholar 

Download references

Acknowledgements

Authors would like to acknowledge Principal and Management of Mepco Schlenk Engineering College, Sivakasi for their encouragement and technical support for this research work. This Project was funded by the Deanship of scientific Research (DsR) at King Abdulaziz University, Jeddah, under grant no. (J: 1-611-1441). The authors, therefore, acknowledge with thanks DSR for technical and financial support.

Author information

Authors and Affiliations

  1. Department of Information Technology, King Abdulaziz University, Jeddah, 21589, Saudi Arabia

    Faris Ketab & Seyed Mohamed Buhari

  2. Department of Electronics and Communication Engineering, Centre for Image Processing and Pattern Recognition, Mepco Schlenk Engineering College, Sivakasi, Tamilnadu, 626005, India

    Newlin Shebiah Russel & Arivazhagan Selvaraj

  3. School of Business, Universiti Teknologi Brunei, Bandar Seri Begawan, Brunei Darussalam

    Seyed Mohamed Buhari

Authors
  1. Faris Ketab
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Newlin Shebiah Russel
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Arivazhagan Selvaraj
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Seyed Mohamed Buhari
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Newlin Shebiah Russel.

Ethics declarations

Conflict of interest

The authors declare that there is no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Ketab, F., Russel, N.S., Selvaraj, A. et al. Parallel deep learning architecture with customized and learnable filters for low-resolution face recognition. Vis Comput 39, 6699–6710 (2023). https://doi.org/10.1007/s00371-022-02757-y

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00371-022-02757-y

Keywords

Search

Navigation