Skip to main content
SpringerLink
Log in

Regularization and attention feature distillation base on light CNN for Hyperspectral face recognition

  • 1182: Deep Processing of Multimedia Data
  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

The hyperspectral imaging, capturing discriminative information across a series of spectrum bands, leads to building a robust face recognition system. Motivated by the success of deep convolutional network and transferring learning, this paper proposed an end-to-end hyperspectral face recognition model based on a light Convolutional Neural Network (CNN) and transfer learning. To boost the performance of hyperspectral face recognition, the Max-Feature-Map (MFM) activation function and fine-tuning (structure regularization(L2SP) or attention feature distillation (AFD)) are introduced to optimize the deep network, which will learn the fine feature representation across different bands. Especially, this method incorporates regularization and AFD cooperation into the transfer learning strategy on the visible face data. By feeding back hyperspectral images to the pretrained light CNN network, we can design an end-to end model that can leverage the generalization ability for hyperspectral face images. Finally, the entire model is trained and verified on the PolyU-HSFD, CMU, and UWA hyperspectral face datasets using the associated standard evaluation protocols. Experimental results demonstrate that the improved Light CNN network can get good representations of hyperspectral face features and the joint training with a combination of L2SP and AFD exhibits better recognition performance than the state-of-the-art methods based on other deep networks.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

References

  1. Allen D (2016) An overview of spectral imaging of human skin toward face recognition. In: Face recognition across the imaging Spectrum. Springer, pp 1–19

  2. Aman G, Hasan D (2018) 3D discrete wavelet transform-based feature extraction for hyperspectral face recognition. IET Biometrics 7(1):49–55

    Article  Google Scholar 

  3. Azizpour H, Razavian A, Sullivan J (2015) From generic to specific deep representations for visual recognition. In: IEEE Confer Comput Vision Pattern Recognition. IEEE, pp 36–45

  4. Bo C, Lu H, Wang D (2018) Spectral-spatial K-nearest neighbor approach for hyperspectral image classification. Multimed Tools Appl 77:10419–10436

    Article  Google Scholar 

  5. Cai Z, Vasconcelos N (2018) Cascade R-CNN: delving into high quality object detection. In: IEEE Conference Comput Vis Pattern Recognition. IEEE, pp 6154–6162

  6. Denes L, Metes P, Liu Y (2002) Hyperspectral face database, Robotics Inst., Pittsburgh, PA, Tech. Rep. CMU-RI-TR-02-25

  7. Di W, Zhang L, Zhang D, Pan Q (2010) Studies on hyperspectral face recognition in visible spectrum with feature band selection. IEEE Trans Syst Man Cybern Syst Hum-  Part A: Systems and Humans 40(6):1354–1361

  8. Guo Y, Zhang L, Hu Y, He X, Gao J (2016) Ms-celeb-1m: a dataset and benchmark for large-scale face recognition. In: Eur Confer Comput Vis. Springer, pp 87–102

  9. He K, Zhang X, Ren S (2016) Deep residual learning for image recognition. In: IEEE Confer Comput Vis Pattern Recognition. IEEE, pp 770–778

  10. Hu J, Shen L, Albanie S (2017) Squeeze-and-excitation networks. In: IEEE Conference on Computer Vision and Pattern Recognition. IEEE,  pp 7132–7141

  11. Jason Y, Jeff C, Bengio Y, Lipson H (2014) How transferable are features in deep neural networks. In : Proceedings of Twenty-Eighth Confer Neural Inform Process Syst. ACM, 3320–3328

  12. Krizhevsky A, Sutskever I, Hinton G (2012) ImageNet classification with deep convolutional neural networks. In: Conference and Workshop on Neural Information Processing Systems. ACM, pp 1097–1105

  13. Kruse F (2002) Comparison of AVIRIS and Hyperion for hyperspectral mineral mapping. In : 11th JPL Airborne Geoscience Workshop

  14. Li Z, Hoiem DD (2017) Learning without forgetting. IEEE Trans Pattern Anal Mach Intell 40(12):2935–2947

    Article  Google Scholar 

  15. Li X, Grandvalet Y, Davoine F (2018) Explicit inductive bias for transfer learning with convolutional networks. In: Thirty-fifth international conference on machine learning. ACM, pp 2825–2834

  16. Li, X, Xiong H, Wang H, Rao Y, Liu L, Huan J (2019) DELTA: deep learning transfer using feature map with attention for convolutional networks. In: Proceedings of Int Confer Learn Represent. ICLR, pp 1–13

  17. Lin M, Chen Q, Yan S (2014) Network in network. In: Proceedings of International Conference on Learning Representation. ICLR, pp 1–12

  18. Mollahosseini A, Chan D, Mahoor M (2016) Going deeper in facial expression recognition using deep neural networks. In: IEEE Winter Conference on Applications of Computer Vision. IEEE, pp1–10

  19. Osia N, Bourlai T (2014) A spectral independent approach for physiological and geometric based face recognition in the visible, middle-wave and long-wave infrared bands. Image Vision Comput 32(11):847–859

    Article  Google Scholar 

  20. Pan SQ, Yang Q (2009) A survey on transfer learning. IEEE Trans Knowl Data Eng 22(10):1345–1359

  21. Pan Z, Healey G, Prasad M, Tromberg B (2003) Face recognition in hyperspectral images. IEEE Trans Pattern Anal Mach Intell 25(12):1552–1560

    Article  Google Scholar 

  22. Patel V, Gopalan R, Li R, Chellappa R (2015) Visual domain adaptation: a survey of recent advances. IEEE Signal Process Mag 32(3):53–69

  23. PolyU-HSFD (1991), www4.comp.polyu.edu.hk/∼biometrics/.

  24. Ren S, He K, Girshick R, Sun J (2015) Faster R-CNN: towards real-time object detection with region proposal networks. In: Twenty-Ninth Confer Neural Inform Process Syst. ACM, pp 91–99

  25. Sharma S, Diba A (2016) Hyperspectral CNN for image classification &band selection, with application to face recognition. Technical report KUL/ESAT/PSI/1604, KU Leuven, Belgium

  26. Simonyan K, Zisserman A (2015) Very deep convolutional networks for large-scale image recognition. In: Proceedings of International Conference on Learning Representation. ICLR2015, pp 1–14

  27. Tan M, Quoc V (2019) Efficient net: rethinking model scaling for convolutional neural networks. In: Int Confer Machine Learn, ACM,  pp 6105–6114

  28. Uzair M, Mahmood A, Mian A (2013) Hyperspectral face recognition using 3D-DCT and partial least squares. In: British Machine Vision Conference. British Machine Vision Association, pp 1–7

  29. Uzair M, Mahmood A, Mian A (2015) Hyperspectral face recognition with spatiospectral information fusion and PLS regression. IEEE Trans Image Process 24(3):1127–1137

    Article  MathSciNet  Google Scholar 

  30. Veera G, Vasuki S (2017) Clustering based band selection for endmember extraction using simplex growing algorithm in hyperspectral images. Multimedia Tools and Applications 76:8355–8371

  31. Wang K, Gao X, Zhao Y, Li X, Dou D, Xu C (2020) Pay attention to features, transfer learn faster CNNs. In: Proceedings of International Conference on Learning Representations. ICLR2020, pp 1–14

  32. Wu X, He R, Sun Z (2018) A light CNN for deep face representation with Noisy labels. IEEE Trans Information Forensics Secur 13(11):2884–2896

    Article  Google Scholar 

  33. Xie Z, Li Y (2019) Hyperspectral face recognition using block based convolution neural network and AdaBoost band selection. In: Proceedings of  International conference on systems and informatics. IEEE, pp 1270–1274

  34. Yi D, Lei Z, Liao S, Li Z (2014) Learning face representation from scratch. arXiv preprint:1411.7923

  35. Yim J, Joo D, Bae J, Kim J (2017) A gift from knowledge distillation: fast optimization, network minimization and transfer learning. In: IEEE Confer Comput Vis Pattern Recognit. IEEE, pp 7130–7138

  36. Zagoruyko S, Komodakis N (2017) Paying more attention to attention: improving the performance of convolutional neural networks via attention transfer. In: Proceedings of  International conference on learning representations. ICLR2017,  pp 1–13

  37. Zhang X, Pan Z, Lu X (2018) Hyperspectral image classification based on joint spectrum of spatial space and spectral space. Multimed Tools Appl 77:29759–29777

    Article  Google Scholar 

  38. Zhao H, Qi X, Shen X, Shi J, Jia J (2018) ICNet for real-time semantic segmentation on high-resolution images. In: Eur Conference Comput Vis. Springer, pp 405–420

Download references

Acknowledgments

This paper is supported by the National Nature Science Foundation of China (No.61861020), Science &Technology Project of Education Bureau of Jiangxi Province (No. GJJ190578), Jiangxi Province Graduate Innovation Special Fund Project (No. YC2020-S571).

Author information

Authors and Affiliations

  1. Key Laboratory of Optic-Electronic and Communication, Jiangxi Science and Technology Normal University, Nanchang, China

    Zhihua Xie, Jieyi Niu & Li Yi

  2. Chester Carlson Center for Imaging Science, Rochester Institute of Technology, Rochester, NY, USA

    Guoyu Lu

Authors
  1. Zhihua Xie
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jieyi Niu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Li Yi
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Guoyu Lu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Zhihua Xie.

Ethics declarations

Conflict of interests

The authors declare that they have no conflict of interest.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Xie, Z., Niu, J., Yi, L. et al. Regularization and attention feature distillation base on light CNN for Hyperspectral face recognition. Multimed Tools Appl 81, 19151–19167 (2022). https://doi.org/10.1007/s11042-021-10537-4

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-021-10537-4

Keywords

Search

Navigation