Skip to main content
SpringerLink
Log in

Scale fusion light CNN for hyperspectral face recognition with knowledge distillation and attention mechanism

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

Hyperspectral imaging technology, combining traditional imaging and spectroscopy technologies to simultaneously acquire spatial and spectral information, is deemed to be an intuitive medium for robust face recognition. However, the intrinsic structure of hyperspectral images is more complicated than ordinary gray-scale or RGB images, how to fully explore the discriminant and correlation features with only a limited number of hyperspectral samples for deep learning training has not been well studied. In response to these problems, this paper proposes an end-to-end multiscale fusion lightweight convolution neural network (CNN) framework for hyperspectral face recognition, termed as the features fusion with channel attention network (FFANet). Firstly, to capture richer subtle details, we introduce Second-Order Efficient Channel Attention (SECA) as the variant of Efficient Channel Attention (ECA) into the framework. The difference from ECA is that SECA can extract the second-order information of each channel to improve the network’s feature extraction ability and is more suitable for the complexity of hyperspectral data. Secondly, we further fuse multiscale information to yield a comprehensive and discriminative representation learning. Finally, the joint of Self-Supervision and Knowledge Distillation (SSKD) is exploited to train an efficient deep model, which can learn more dark knowledge from the trained teacher network. The experimental results on three benchmark hyperspectral face databases of PolyU, CMU, and UWA show that the proposed approach has achieved competitive accuracy and efficiency on the basis of significantly reducing the storage space and computation overheads. These characteristics also show its wide applicability on edge/mobile devices.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

References

  1. Liang H, Gao J, Qiang N (2021) A novel framework based on wavelet transform and principal component for face recognition under varying illumination. Appl Intell 51:1762–1783

    Article  Google Scholar 

  2. Li Y, Xie T, Wang P, Wang J, Liu S, Zhou X, Zhang X (2018) Joint spectral-spatial hyperspectral image classification based on hierarchical subspace switch ensemble learning algorithm. Appl Intell 48:4128–4148

    Article  Google Scholar 

  3. Osia N, Bourlai T (2014) A spectral independent approach for physiological and geometric based face recognition in the visible, middle-wave and long-wave infrared bands. Image Vision Comput 32:847–859

    Article  Google Scholar 

  4. Uzair M, Mahmood A, Mian A (2015) Hyperspectral face recognition with spatio-spectral information fusion and PLS regression. IEEE Trans Image Process 24:1127–1137

    Article  MathSciNet  Google Scholar 

  5. Wei D, Zhang L, Hu N, Liu L, Ma N, Zhao Y (2017) Hyperspectral face recognition with spatial-spectral fusion information and gabor feature. Trans Beijing Inst Technol 37:1077–1083

    Google Scholar 

  6. Uzair M, Mahmood A, Mian A (2013) Hyperspectral face recognition using 3d-DCT and partial least squares. In: Proceedings of the British machine vision conference, pp 1–7

  7. Shen L, Zheng S (2012) Hyperspectral face recognition using 3D Gabor wavelets. In: Proceedings of the international conference on pattern recognition, pp 1574–1577

  8. Liang J, Zhou J, Gao Y (2015) 3D local derivative pattern for hyperspectral face recognition. In: IEEE international conference and workshops on automatic face and gesture recognition, pp 1–6

  9. Aman G, Hasan D (2018) 3D Discrete wavelet transform-based feature extraction for hyperspectral face recognition. IET Biomet 7:49–55

    Article  Google Scholar 

  10. Pan Z, Healey G, Tromberg B (2009) Comparison of Spectral-Only and Spectral/Spatial face recognition for personal identity verification EURASIP. Journal Advances In Signal Processing

  11. Xie Z, Li Y, Niu J, Shi L (2019) Hyperspectral face recognition using block based convolution neural network and AdaBoost band selection. Int Conf Syst Inform 1270–1274

  12. Di W, Zhang L, Zhang D, Pan Q (2010) Studies on hyperspectral face recognition in visible spectrum with feature band selection. IEEE Trans Syst Man Cybern Part A Syst Humans 40:1354–1361

    Article  Google Scholar 

  13. Li Y, Wu Y, Zhang N (2018) An image-level classification framework for Hyperspectral image with CNNs. In: Proceedings of 14th IEEE international conference on signal processing (ICSP), pp 586–590

  14. Xie Z, Niu J, Li Y (2020) Hyperspectral face recognition based on SLRC for single sample problem. Proc SPIE 11428:114280Q

    Google Scholar 

  15. Appice A, Guccione P, Acciaro E, Malerba D (2020) Detecting salient regions in a bi-temporal hyperspectral scene by iterating clustering and classification. Appl Intell 50:3179–3200

    Article  Google Scholar 

  16. Taherkhani F, Dawson J, Nasrabadi N Prasad S, Chanussot J (eds) (2020) Deep sparse band selection for hyperspectral face recognition. Springer, Cham

  17. Hsieh T, Kiang F (2020) Comparison of CNN algorithms on hyperspectral image classification in agricultural lands. Sensor 20:1734

    Article  Google Scholar 

  18. Wu X, He R, Sun Z (2018) A light CNN for deep face representation with noisy labels. In: IEEE transactions on information forensics and security, vol 13, pp 2884–2896

  19. Wang Q, Wu B, Zhu P, Li P (2020) ECA-Net: efficient channel attention for deep convolutional neural networks. In: IEEE conference on computer vision and pattern recognition, pp 11534–11542

  20. Larochelle H, Hinton GE (2010) Learning to combine foveal glimpses with a third-order boltzmann machine. In: Proceedings of the 23rd international conference on neural information processing systems, pp 143–1251

  21. Hu J, Shen L, Albanie S (2017) Squeeze-and-excitation Networks. In: IEEE conference on computer vision and pattern recognition, pp 7132–7141

  22. Gardner MW, Dorling SR (1998) Artificial neural networks (the multilayer perceptron)—a review of applications in the atmospheric sciences. Atmospheric Environ 32:2627–2636

    Article  Google Scholar 

  23. Woo S, Park J, Lee J, Kweon IS (2018) CBAM: Convolutional Block attention module. In: European conference on computer vision, pp 3–19

  24. Hu J, Shen L, Albanie S, Sun G (2018) Gather-excite: Exploiting feature context in convolutional neural networks. In: Proceedings of international conference on neural information processing system, pp 1–11

  25. Chollet F (2017) Xception: Deep learning with depthwise separable convolutions. In: IEEE conference on computer vision and pattern recognition, pp 1251–1258

  26. Li X, Lei L, Sun Y, Li M, Kuang G (2020) Multimodal bilinear fusion network with Second-Order Attention-Based channel selection for land cover classification. IEEE J Select Topics Appl Earth Observ Remote Sens 13:1011–1026

    Article  Google Scholar 

  27. Roy AG, Navab N, Wachinge C (2019) Recalibrating fully convolutional networks with spatial and channel squeeze and excitation blocks. IEEE Trans Med Imaging 38:540–549

    Article  Google Scholar 

  28. Hinton G, Vinyals O, Dean J (2015) Distilling the knowledge in a neural network NIPS. Deep Learning Workshop

  29. Zagoruyko S, Komodakis N (2017) Paying more attention to attention: Improving the performance of convolutional neural networks via attention transfer. International Conference on Learning Representations

  30. Yim J, Joo D, Bae J, Kim J (2017) A gift from knowledge distillation: Fast optimization, network minimization and transfer learning. In: IEEE conference on computer vision and pattern recognition, pp 4133–4141

  31. Kim J, Park S, Kwak N (2018) Paraphrasing complex network: Network compression via factor transfer. Neural Inform Process Syst 2760–2769

  32. Liu Y, Cao J, Li B, Yuan C, Hu W, Li Y, Duan Y (2019) Knowledge distillation via instance relationship graph. IEEE Conf Comput Vision Pattern Recognit 7096–7104

  33. Xu G, Liu Z, Li X, Loy C (2020) Knowledge distillation meets Self-Supervision. European Conf Comput Vision 588–604

  34. Gao Z, Xie J, Wang Q, Li P (2019) Global second-order pooling convolutional networks. IEEE Conf Comput Vision Pattern Recognit 3024–3033

  35. Lin M, Chen Q, Yan S (2014) Network in network. International Conference on Learning Representation

  36. Simonyan K, Zisserman A (2015) Very deep convolutional networks for large-scale image recognition. International Conference on Learning Representation

  37. Mu G, Huang D, Hu G, Sun J, Wang Y (2019) Led3d: A Lightweight and Efficient Deep Approach to Recognizing Low-quality 3D Faces. IEEE Conf Comput Vision Pattern Recognit 5773–5782

  38. PolyU-HSFD (2010) http://www4.comp.polyu.edu.hk/biometrics

  39. Denes L, Metes P, Liu Y (2002) Hyperspectral face database, Robotics Inst Pittsburgh, PA, Tech. Rep CMU-RI-TR-02-25

  40. Yi D, Lei Z, Liao S, Li SZ (2014) Learning face representation from scratch, arXiv:1411.7923

  41. Wang K, Gao X, Zhao Y, Li X, Dou D, Xu C-Z (2020) Pay attention to features, transfer learn faster CNNs. In: International conference on learning representations

  42. Robila S (2008) Toward hyperspectral face recognition. Image Processing: Algorithms and Systems VI 6812

  43. Krizhevsky A, Sutskever I, Hinton G (2012) ImageNet classification with deep convolutional neural networks. In: Conference and workshop on neural information processing systems, p 25

  44. mollahosseini A, chan D, mahoor MH (2016) Going deeper in facial expression recognition using deep neural networks. In: IEEE winter conference on applications of computer vision, pp 1–10

  45. He K, Zhang X, Ren S (2016) Deep residual learning for image recognition. In: IEEE conference on computer vision and pattern recognition, pp 770–778

  46. Howard A, Sandler M, Chu G, Chen L, et al. (2019) Searching for MobileNet v3. IEEE Int Conf Comput Vision 2019:1314–1324

    Google Scholar 

  47. Ma N, Zhang X, Zheng H, Sun J (2018) ShuffleNet v2: Practical Guidelines for efficient CNN architecture design. European Conf Comput Vision 2018:116–131

    Google Scholar 

  48. Chen S, Liu Y, Gao X, Han Z (2018) Mobilefacenets: efficient CNNs for accurate real-time face verification on Mobile Devices. Chinese Conf Biomet Recognit 2018:428–438

    Google Scholar 

  49. Zhao M, Jia Z, Cai Y, Chen X, Gong D (2021) Advanced variations of two-dimensional principal component analysis for face recognition. Neurocomputing, 446, In Press

  50. Wu F, Jing X, Dong X, Hu R, Yue D, Wang L, Wang R, Guo C (2020) Intraspectrum discrimination and interspectrum correlation analysis deep network for multispectral face recognition. IEEE Trans Cybern 45(2):242–252

    Google Scholar 

Download references

Acknowledgements

This research is supported by the National Nature Science Foundation of China (No.61861020), Science & Technology Project of Education Bureau of Jiangxi Province (No. GJJ190578)

Author information

Authors and Affiliations

  1. Key Lab of Optic-Electronic and Communication, Jiangxi Science and Technology Normal University, Nanchang, China

    Jie-Yi Niu, Zhi-Hua Xie, Yi Li, Si-Jia Cheng & Jia-Wei Fan

Authors
  1. Jie-Yi Niu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Zhi-Hua Xie
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yi Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Si-Jia Cheng
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Jia-Wei Fan
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Zhi-Hua Xie.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Niu, JY., Xie, ZH., Li, Y. et al. Scale fusion light CNN for hyperspectral face recognition with knowledge distillation and attention mechanism. Appl Intell 52, 6181–6195 (2022). https://doi.org/10.1007/s10489-021-02721-8

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-021-02721-8

Keywords

Search

Navigation