Skip to main content

Advertisement

Springer Nature Link
Log in

Leaning compact and representative features for cross-modality person re-identification

  • Published:
World Wide Web Aims and scope Submit manuscript

Abstract

This paper pays close attention to the cross-modality visible-infrared person re-identification (VI Re-ID) task, which aims to match pedestrian samples between visible and infrared modes. In order to reduce the modality-discrepancy between samples from different cameras, most existing works usually use constraints based on Euclidean metric. Because of the Euclidean based distance metric strategy cannot effectively measure the internal angles between the embedded vectors, the existing solutions cannot learn the angularly discriminative feature embedding. Since the most important factor affecting the classification task based on embedding vector is whether there is an angularly discriminative feature space, in this paper, we present a new loss function called Enumerate Angular Triplet (EAT) loss. Also, motivated by the knowledge distillation, to narrow down the features between different modalities before feature embedding, we further present a novel Cross-Modality Knowledge Distillation (CMKD) loss. Benefit from the above two considerations, the embedded features are discriminative enough in a way to tackle modality-discrepancy problem. The experimental results on RegDB and SYSU-MM01 datasets have demonstrated that the proposed method is superior to the other most advanced methods in terms of impressive performance. Code is available at https://github.com/IVIPLab/LCCRF.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

References

  1. Chen, W., Chen, X., Zhang, J., Huang, K.: Beyond triplet loss: a deep quadruplet network for person re-identification. In: Proceedings of the IEEE conference on Computer Vision and Pattern Recognition, pp. 403–412 (2017)

  2. Choi, S., Lee, S., Kim, Y., Kim, T., Kim, C.: Hi-cmd: hierarchical cross-modality disentanglement for visible-infrared person re-identification. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 10257–10266 (2020)

  3. Dai, P., Ji, R., Wang, H., Wu, Q., Huang, Y.: Cross-modality person re-identification with generative adversarial training. In: International Joint Conference on Artificial Intelligence, pp. 677–683 (2018)

  4. Fan, X., Luo, H., Zhang, C., Jiang, W.: Cross-spectrum dual-subspace pairing for rgb-infrared cross-modality person re-identification. arXiv:2003.00213 (2020)

  5. Fan, X., Jiang, W., Luo, H., Fei, M.: Spherereid: Deep hypersphere manifold embedding for person re-identification. Journal of Visual Communication and Image Representation 60, 51–58 (2019)

    Article  Google Scholar 

  6. Feng, Z., Lai, J., Xie, X.: Learning view-specific deep networks for person re-identification. IEEE Transactions on Image Processing 27(7), 3472–3483 (2018)

    Article  MathSciNet  Google Scholar 

  7. Feng, Z., Lai, J., Xie, X.: Learning modality-specific representations for visible-infrared person re-identification. IEEE Transactions on Image Processing 29, 579–590 (2020)

    Article  MathSciNet  Google Scholar 

  8. Gao, G., Yu, Y., Yang, J., Qi, G.J., Yang, M.: Hierarchical deep cnn feature set-based representation learning for robust cross-resolution face recognition. IEEE Transactions on Circuits and Systems for Video Technology (2020)

  9. Gao, G., Yang, J., Jing, X.Y., Shen, F., Yang, W., Yue, D.: Learning robust and discriminative low-rank representations for face recognition with occlusion. Pattern Recognition 66, 129–143 (2017)

    Article  Google Scholar 

  10. Gao, G., Yu, Y., Xie, J., Yang, J., Yang, M., Zhang, J.: Constructing multilayer locality-constrained matrix regression framework for noise robust face super-resolution. Pattern Recognition 110, 107539 (2021)

    Article  Google Scholar 

  11. Hao, Y., Wang, N., Li, J., Gao, X.: Hsme: hypersphere manifold embedding for visible thermal person re-identification. In: Proceedings of the AAAI conference on Artificial Intelligence, pp. 8385–8392 (2019)

  12. Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv:1412.6980 (2014)

  13. Leng, Q., Ye, M., Tian, Q.: A survey of open-world person re-identification. IEEE Transactions on Circuits and Systems for Video Technology 30(4), 1092–1108 (2020)

    Article  Google Scholar 

  14. Li, R., Zhang, B., Kang, D.J., Teng, Z.: Deep attention network for person re-identification with multi-loss. Computers & Electrical Engineering 79, 106455 (2019)

    Article  Google Scholar 

  15. Liu, H., Shi, W., Huang, W., Guan, Q.: A discriminatively learned feature embedding based on multi-loss fusion for person search. In: Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 1668–1672. IEEE (2018)

  16. Liu, J., Zha, Z.J., Tian, Q., Liu, D., Yao, T., Ling, Q., Mei, T.: Multi-scale triplet cnn for person re-identification. In: Proceedings of the ACM international conference on Multimedia, pp. 192–196 (2016)

  17. Liu, H., Feng, J., Qi, M., Jiang, J., Yan, S.: End-to-end comparative attention networks for person re-identification. IEEE Transactions on Image Processing 26(7), 3492–3506 (2017)

    Article  MathSciNet  Google Scholar 

  18. Liu, H., Cheng, J., Wang, W., Su, Y., Bai, H.: Enhancing the discriminative feature learning for visible-thermal cross-modality person re-identification. Neurocomputing 398, 11–19 (2020)

    Article  Google Scholar 

  19. Lu, Y., Wu, Y., Liu, B., Zhang, T., Li, B., Chu, Q., Yu, N.: Cross-modality person re-identification with shared-specific feature transfer. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13379–13389 (2020)

  20. Lu, H., Zhang, M., Xu, X., Li, Y., Shen, H.T.: Deep fuzzy hashing network for efficient image retrieval. IEEE Transactions on Fuzzy Systems 29(1), 166–176 (2020)

    Article  Google Scholar 

  21. Luo, H., Jiang, W., Gu, Y., Liu, F., Liao, X., Lai, S., Gu, J.: A strong baseline and batch normalization neck for deep person re-identification. IEEE Transactions on Multimedia 22(10), 2597–2609 (2019)

    Article  Google Scholar 

  22. Nguyen, D.T., Hong, H.G., Kim, K.W., Park, K.R.: Person recognition system based on a combination of body images from visible light and thermal cameras. Sensors 17(3), 605 (2017)

    Article  Google Scholar 

  23. Qian, X., Fu, Y., Xiang, T., Wang, W., Qiu, J., Wu, Y., Jiang, Y.G., Xue, X.: Pose-normalized image generation for person re-identification. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 650–667 (2018)

  24. Radenović, F., Tolias, G., Chum, O.: Fine-tuning cnn image retrieval with no human annotation. IEEE Transactions on Pattern Analysis and Machine Intelligence 41(7), 1655–1668 (2019)

    Article  Google Scholar 

  25. Serikawa, S., Lu, H.: Underwater image dehazing using joint trilateral filter. Computers & Electrical Engineering 40(1), 41–50 (2014)

    Article  Google Scholar 

  26. Sun, Y., Zheng, L., Yang, Y., Tian, Q., Wang, S.: Beyond part models: Person retrieval with refined part pooling (and a strong convolutional baseline). In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 480–496 (2018)

  27. Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., Wojna, Z.: Rethinking the inception architecture for computer vision. In: Proceedings of the IEEE conference on Computer Vision and Pattern Recognition, pp. 2818–2826 (2016)

  28. Wang, X., Girshick, R., Gupta, A., He, K.: Non-local neural networks. In: Proceedings of the IEEE conference on Computer Vision and Pattern Recognition, pp. 7794–7803 (2018)

  29. Wang, Z., Wang, Z., Zheng, Y., Chuang, Y.Y., Satoh, S.: Learning to reduce dual-level discrepancy for infrared-visible person re-identification. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 618–626 (2019)

  30. Wang, G., Yuan, Y., Chen, X., Li, J., Zhou, X.: Learning discriminative features with multiple granularities for person re-identification. In: Proceedings of the ACM international conference on Multimedia, pp. 274–282 (2018)

  31. Wang, G., Zhang, T., Cheng, J., Liu, S., Yang, Y., Hou, Z.: Rgb-infrared cross-modality person re-identification via joint pixel and feature alignment. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 3623–3632 (2019)

  32. Wang, G.A., Zhang, T., Yang, Y., Cheng, J., Chang, J., Liang, X., Hou, Z.G.: Cross-modality paired-images generation for rgb-infrared person re-identification. In: Proceedings of the AAAI Conference on Artificial Intelligence, pp. 12144–12151 (2020)

  33. Wei, Z., Yang, X., Wang, N., Gao, X.: Flexible body partition-based adversarial learning for visible infrared person re-identification. IEEE Transactions on Neural Networks and Learning Systems (2021)

  34. Wei, L., Zhang, S., Gao, W., Tian, Q.: Person transfer gan to bridge domain gap for person re-identification. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 79–88 (2018)

  35. Wojke, N., Bewley, A.: Deep cosine metric learning for person re-identification. In: Proceedings of the IEEE Winter Conference on Applications of Computer Vision (WACV), pp. 748–756. IEEE (2018)

  36. Wu, A., Zheng, W.S., Yu, H.X., Gong, S., Lai, J.: Rgb-infrared cross-modality person re-identification. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 5380–5389 (2017)

  37. Wu, A., Zheng, W.S., Gong, S., Lai, J.: Rgb-ir person re-identification by cross-modality similarity preservation. International Journal of Computer Vision 128(6), 1765–1785 (2020)

    Article  MathSciNet  Google Scholar 

  38. Xu, X., Wang, T., Yang, Y., Zuo, L., Shen, F., Shen, H.T.: Cross-modal attention with semantic consistence for image-text matching. IEEE Transactions on Neural Networks and Learning Systems 31(12), 5412–5425 (2020)

    Article  Google Scholar 

  39. Ye, M., Lan, X., Leng, Q.: Modality-aware collaborative learning for visible thermal person re-identification. In: Proceedings of the ACM International Conference on Multimedia, pp. 347–355 (2019)

  40. Ye, M., Lan, X., Li, J., Yuen, P.: Hierarchical discriminative learning for visible thermal person re-identification. In: Proceedings of the AAAI Conference on Artificial Intelligence, pp. 7501–7508 (2018)

  41. Ye, M., Shen, J., Lin, G., Xiang, T., Shao, L., Hoi, S.C.: Deep learning for person re-identification: A survey and outlook. IEEE Transactions on Pattern Analysis and Machine Intelligence (2021)

  42. Ye, M., Wang, Z., Lan, X., Yuen, P.C.: Visible thermal person re-identification via dual-constrained top-ranking. In: International Joint Conference on Artificial Intelligence, vol. 1, p. 2 (2018)

  43. Ye, M., Lan, X., Leng, Q., Shen, J.: Cross-modality person re-identification via modality-aware collaborative ensemble learning. IEEE Transactions on Image Processing 29, 9387–9399 (2020)

    Article  Google Scholar 

  44. Ye, M., Lan, X., Wang, Z., Yuen, P.C.: Bi-directional center-constrained top-ranking for visible thermal person re-identification. IEEE Transactions on Information Forensics and Security 15, 407–419 (2020)

    Article  Google Scholar 

  45. Ye, M., Shen, J., Shao, L.: Visible-infrared person re-identification via homogeneous augmented tri-modal learning. IEEE Transactions on Information Forensics and Security 16, 728–739 (2020)

    Article  Google Scholar 

  46. Ye, H., Liu, H., Meng, F., Li, X.: Bi-directional exponential angular triplet loss for rgb-infrared person re-identification. IEEE Transactions on Image Processing 30, 1583–1595 (2021)

    Article  Google Scholar 

  47. Zhang, J., Xu, X., Shen, F., Lu, H., Liu, X., Shen, H.T.: Enhancing audio-visual association with self-supervised curriculum learning. In: Proceedings of the AAAI Conference on Artificial Intelligence, pp. 3351–3359 (2021)

  48. Zhao, Y.B., Lin, J.W., Xuan, Q., Xi, X.: Hpiln: a feature learning framework for cross-modality person re-identification. IET Image Processing 13(14), 2897–2904 (2019)

    Article  Google Scholar 

  49. Zhao, C., Lv, X., Zhang, Z., Zuo, W., Wu, J., Miao, D.: Deep fusion feature representation learning with hard mining center-triplet loss for person re-identification. IEEE Transactions on Multimedia 22(12), 3180–3195 (2020)

    Article  Google Scholar 

  50. Zhao, C., Wang, X., Zuo, W., Shen, F., Shao, L., Miao, D.: Similarity learning with joint transfer constraints for person re-identification. Pattern Recognition 97, 107014 (2020)

    Article  Google Scholar 

  51. Zheng, W.S., Li, X., Xiang, T., Liao, S., Lai, J., Gong, S.: Partial person re-identification. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 4678–4686 (2015)

  52. Zheng, L., Yang, Y., Hauptmann, A.G.: Person re-identification: Past, present and future. arXiv:1610.02984 (2016)

  53. Zhong, Z., Zheng, L., Kang, G., Li, S., Yang, Y.: Random erasing data augmentation. In: Proceedings of the AAAI Conference on Artificial Intelligence, pp. 13001–13008 (2020)

Download references

Acknowledgements

This work was supported in part by the National Key Research and Development Program of China under Project nos. 2018AAA0100102 and 2018AAA0100100, the National Natural Science Foundation of China under Grant nos. 61972212, 61772568 and 61833011, the Natural Science Foundation of Jiangsu Province under Grant no. BK20190089, the Six Talent Peaks Project in Jiangsu Province under Grant no. RJFW-011, Youth science and technology innovation talent of Guangdong Special Support Program, and Open Fund Project of Provincial Key Laboratory for Computer Information Processing Technology (Soochow University) (No. KJS1840).

Author information

Authors and Affiliations

  1. Institute of Advanced Technology, Nanjing University of Posts and Telecommunications, Nanjing, China

    Guangwei Gao, Hao Shao & Fei Wu

  2. Digital Content and Media Sciences Research Division, National Institute of Informatics, Tokyo, Japan

    Guangwei Gao & Yi Yu

  3. Key Laboratory of Machine Intelligence and Advanced Computing, Ministry of Education, Sun Yat-sen University, Guangzhou, China

    Meng Yang

Authors
  1. Guangwei Gao
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Hao Shao
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Fei Wu
    View author publications

    You can also search for this author inPubMed Google Scholar

  4. Meng Yang
    View author publications

    You can also search for this author inPubMed Google Scholar

  5. Yi Yu
    View author publications

    You can also search for this author inPubMed Google Scholar

Ethics declarations

Conflicts of interest

The authors declare that they have no conflict of interest.

Additional information

Guangwei Gao and Hao Shao contributed equally to this work.

This article belongs to the Topical Collection: Special Issue on Synthetic Media on the Web

Guest Editors: Huimin Lu, Xing Xu, Jože Guna, and Gautam Srivastava

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Gao, G., Shao, H., Wu, F. et al. Leaning compact and representative features for cross-modality person re-identification. World Wide Web 25, 1649–1666 (2022). https://doi.org/10.1007/s11280-022-01014-5

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11280-022-01014-5

Keywords