Skip to main content
SpringerLink
Log in

Local Feature for Visible-Thermal PReID Based on Transformer

  • Conference paper
  • First Online:
Intelligent Computing Theories and Application (ICIC 2022)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 13393))

Included in the following conference series:

  • 1555 Accesses

Abstract

Person re-identification based on infrared image and RGB image is a cross-modality pedestrian recognition, which is a challenging task. The traditional goal of person re-identification is to find a given person’s image from an image database, often from a single modality database. In real applications, there are often multiple modalities of data. Traditional single modality tasks have limitations. Cross-modality person re-identification needs to extract features from RGB and infrared images. In our work, we take advantage of both global and local features. First, we use a dual-path VIT structure to extract features from RGB images and infrared images, respectively. Secondly, we cut the local features in the spatial direction and input the shared VIT layer to learn the local features. The loss function consists of Identity loss, Triplet loss, and Center loss. The model can capture shared features between modality and improve cross-modality similarity. Finally, we performed experiments on two datasets, SYSU-MM01 and RegDB, and compared them with other methods in recent studies.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 99.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 129.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Zheng, L., Yang, Y., Hauptmann, A.G.: Person re-identification: past, present and future. arXiv preprint arXiv:1610.02984 (2016)

  2. Ye, M.: Deep learning for person re-identification: a survey and outlook (2020)

    Google Scholar 

  3. Gheissari, N., Sebastian, T.B., Hartley, R.: Person reidentification using spatiotemporal appearance. In: 2006 IEEE Computer Society Conference on IEEE Computer Vision and Pattern Recognition, pp. 1528–1535 (2006)

    Google Scholar 

  4. Bazzani, L., Cristani, M., Perina, A., et al.: Multiple-shot person re-identification by HPE signature. In: 2010 20th International Conference on Pattern Recognition, pp. 1413–1416. IEEE (2010)

    Google Scholar 

  5. Farenzena, M., Bazzani, L., Perina, A., et al.: Person re-identification by symmetry-driven accumulation of local features. In: 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 2360–2367. IEEE (2010)

    Google Scholar 

  6. Wu, Y., Qin, X., Pan, Y., et al.: Convolution neural network based transfer learning for classifition of floers. In: 2018 IEEE 3rd International Conference on Signal and Image Processing (ICSIP), pp. 562–566. IEEE (2018)

    Google Scholar 

  7. Zagoruyko, S., Komodakis, N.: Learning to compare image patches via convolutional neural networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Pp. 4353–4361 (2015)

    Google Scholar 

  8. Yuan, C., Wu, Y., Qin, X., et al.: An effective image classification method for shallow densely connected convolution networks through squeezing and splitting techniques. Appl. Intell. 49(10), 3570–3586 (2019)

    Article  Google Scholar 

  9. Liu, C., Gong, S., Loy, C.C., Lin, X.: Person re-identification: what features are important? In: Fusiello, A., Murino, V., Cucchiara, R. (eds.) ECCV 2012. LNCS, vol. 7583, pp. 391–401. Springer, Heidelberg (2012). https://doi.org/10.1007/978-3-642-33863-2_39

    Chapter  Google Scholar 

  10. Wu, Y., Zhang, K., Wu, D., Wu, Y., et al.: Person reidentification by multiscale feature representation learning with random batch feature mask. IEEE Trans. Cogn. Dev. Syst. 13(4), 865–874 (2021)

    Google Scholar 

  11. Wu, Y., et al.: Position Attention-Guided Learning for Infrared-Visible Person Re-identification. In: Huang, De-Shuang., Bevilacqua, Vitoantonio, Hussain, Abir (eds.) ICIC 2020. LNCS, vol. 12463, pp. 387–397. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-60799-9_34

    Chapter  Google Scholar 

  12. Dai, P., Ji, R., Wang, H., Wu, Q., Huang, Y.: Cross modality person re-identification with generative adversarial training. In: IJCAI, pp. 677–683 (2018)

    Google Scholar 

  13. Wu, A.: RGB infrared cross modality person re identification. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 5380–5389 (2017)

    Google Scholar 

  14. Wang, G., Zhang, T., Cheng, J., Liu, S., Yang, Y., Hou, Z.: RGB-infrared cross-modality person reidentification via joint pixel and feature alignment. In: ICCV, pp. 3622–3631 (2019)

    Google Scholar 

  15. He, S., Luo, H., Wang, P., Wang, F., Li, H., Jiang, W.: TransReID: transformer-based object reidentification. CoRR, abs/2102.04378 (2021)

    Google Scholar 

  16. Wu, A., Zheng, W.S., Yu, H.X., et al.: RGB-infrared cross-modality person re-identification. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 5380–5389 (2017)

    Google Scholar 

  17. Ye, M., Lan, X., Li, J., et al.: Hierarchical discriminative learning for visible thermal person re-identification. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 32, no. 1 (2018)

    Google Scholar 

  18. Dai, P., Ji, R., Wang, H., et al.: Cross-modality person re-identification with generative adversarial training. In: IJCAI, vol. 1, p. 2 (2018)

    Google Scholar 

  19. Liu, H., Cheng, J., Wang, W., et al.: Enhancing the discriminative feature learning for visible-thermal cross-modality person re-identification. Neurocomputing 398, 11–19 (2020)

    Article  Google Scholar 

  20. Zhang, Z., Zhang, H., Liu, S., et al.: Part-guided graph convolution networks for person re-identification. Pattern Recogn. 120, 108155 (2021)

    Article  Google Scholar 

  21. Vaswani, A., et al.: 2017. Attention is all you need. In NIPS, pp. 5998–6008 (2017)

    Google Scholar 

  22. Dosovitskiy, A., et al.: An image is worth 16x16 words: transformers for image recognition at scale. In ICLR (2021)

    Google Scholar 

  23. Dat, T.N., Hyung, G.H., Ki, W.K.: Person recognition system based on a combination of body images from visible light and thermal cameras. Sensors 17(3), 605 (2017)

    Article  Google Scholar 

  24. Ye, M., Lan, X., Li, J.: Hierarchical discriminative learning for visible thermal person re identification. In: Thirty Second AAAI Conference on Artificial Intelligence (2018)

    Google Scholar 

  25. Ye, M., Wang, Z., Lan, X., Yuen, P.C.: Visible thermal person re-identification via dualconstrained top-ranking. In: Proceedings of International Joint Conference on Artificial Intelligence, pp. 1092–1099 (2018)

    Google Scholar 

  26. Wang, Z., Wang, Z., Zheng, Y.: Learning to reduce dual level discrepancy for infrared visible person re identification. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 618–626 (2019)

    Google Scholar 

  27. Jiang, J., et al.: A cross-modal multi-granularity attention network for RGB-IR person re-identification. Neurocomputing, p. 406 (2020)

    Google Scholar 

  28. Wang, G.-A., Zhang, T., Yang, Y.: Cross-modality paired images generation for RGB–infrared person re–identification. In: Thirty-Fourth AAAI Conference on Artificial Intelligence (2020)

    Google Scholar 

  29. Li, D., Wei, X., Hong, X., Gong, Y.: Infrared-visible cross-modal person re-identification with an X modality. In: Thirty-Fourth AAAI Conference on Artificial Intelligence (2020)

    Google Scholar 

  30. Ye, M., Lan, X., Leng, Q., Shen, J.: Cross-modality person reidentification via modality-aware collaborative ensemble learning. IEEE Trans. Image Process. 29, 9387–9399 (2020)

    Google Scholar 

  31. Hao, Y., Wang, N., Gao, X., Li, J., Wang, X.: Dual-alignment feature embedding for cross-modality person re-identification. In: Proceedings of the 27th ACM International Conference on Multimedia, pp. 57–65 (2019)

    Google Scholar 

  32. Tian, X., Zhang, Z., Lin, S., et al.: Farewell to mutual information: variational distillation for cross-modal person re-identification. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 1522–1531 (2021)

    Google Scholar 

  33. Tian, X., et al.: Farewell to mutual information: variational distillation for cross-modal person re-identification. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 1522–1531 (2021)

    Google Scholar 

  34. Chen, Y., et al.: Neural feature search for RGB-infrared person reidentification. arXiv preprint arXiv:2104.02366 (2021)

  35. Huang, J., Huang, D.S.: Deep reinforcement learning based trajectory pricing on ride-hailing platforms. ACM Trans. Intell. Syst. Technol. vol. 13, no. 3, Article 41 (2022)

    Google Scholar 

  36. Wu, Y., et al.: Person reidentification by multiscale feature representation learning with random batch feature mask. IEEE Trans. Cogn. Dev. Syst. 13(4), 865–874 (2021)

    Article  Google Scholar 

  37. Wu, D., Wang, C., Wu, Y., Wang, Q.-C., Huang, D.S.: Attention deep model with multi-scale deep supervision for person re-identification. IEEE Trans. Emerg. Top. Comput. Intell. 5(1), 70–78 (2021)

    Article  Google Scholar 

  38. Liang, X., Wu, D., Huang, D.S.: Image co-segmentation via locally biased discriminative clustering. IEEE Trans. Knowl. Data Eng. 31(11), 2228–2233 (2019)

    Article  Google Scholar 

  39. Wu, D., et al.: Deep learning based methods for person re-identification: a comprehensive review. Neurocomputing 337, 354–371 (2019)

    Article  Google Scholar 

  40. Wu, D., et al.: Random occlusion-recovery for person re-identification. J. Imaging Sci. Technol. 63(3), 30405-1–30405-9(9) (2019)

    Google Scholar 

  41. Li, B., Fan, Z.T., Zhang, X.L., Huang, D.S.: Robust dimensionality reduction via feature space to feature space distance metric learning. Neural Netw. 112(4), 1–14 (2019)

    Article  MATH  Google Scholar 

  42. Wu, D., et al.: Omnidirectional feature learning for person re-identification. IEEE Access 7, 28402–28411 (2019)

    Article  Google Scholar 

  43. Wu, D., Zheng, S.-J., Yuan, C.-A., Huang, D.S.: A deep model with combined losses for person re-identification. Cogn. Syst. Res. 54, 74–82 (2019)

    Article  Google Scholar 

  44. Wu, D., Zheng, S.-J., Bao, W.-Z., Zhang, X.-P., Yuan, C.-A., Huang, D.S.: A novel deep model with multi-loss and efficient training for person re-identification. Neurocomputing 324, 69–76 (2019)

    Google Scholar 

  45. Peng, C., Zou, L., Huang, D.S.: Discovery of relationships between long non-coding RNAs and genes in human diseases based on tensor completion. IEEE Access 6, 59152–59162 (2018)

    Article  Google Scholar 

  46. Yang, B., Bao, W., Huang, D.S., Chen, Y.: Inference of large-scale time-delayed gene regulatory network with parallel mapReduce cloud platform. Sci. Rep. 8, 17787 (2018)

    Article  Google Scholar 

  47. Shen, Z., Bao, W.-Z., Huang, D.S.: Recurrent neural network for predicting transcription factor binding sites. Sci. Rep. 8, 15270 (2018)

    Article  Google Scholar 

  48. Liu, B., Weng, F., Huang, D.S., Chou, K.-C.: HSCVFNT: inference of time-delayed gene regulatory network based on complex-valued flexible neural tree model. Int. J. Mol. Sci. 19(10), 3178 (2018)

    Article  Google Scholar 

  49. Zhang, H., Zhu, L., Huang, D.S.: DiscMLA: an efficient discriminative motif learning algorithm over high-throughput datasets. IEEE/ACM Trans. Comput. Biol. Bioinf. 15(6), 1810–1820 (2018)

    Article  Google Scholar 

Download references

Acknowledgements

This work was supported by the grant of National Key R&D Program of China (No. 2018AAA0100100 & 2018YFA0902600) and partly supported by National Natural Science Foundation of China (Grant nos. 61732012, 62002266, 61932008, and 62073231), and Introduction Plan of High-end Foreign Experts (Grant no. G2021033002L) and, respectively, supported by the Key Project of Science and Technology of Guangxi (Grant no. 2021AB20147), Guangxi Natural Science Foundation (Grant nos. 2021JJA170204 & 2021JJA170199) and Guangxi Science and Technology Base and Talents Special Project (Grant nos. 2021AC19354 & 2021AC19394).

Author information

Authors and Affiliations

  1. Institute of Machine Learning and Systems Biology, School of Electronics and Information Engineering, Tongji University, Shanghai, 201804, China

    Quanyi Pu

  2. Guangxi Academy of Science, Nanning, 530007, China

    Changan Yuan

  3. Guangxi Key Lab of Human-Machine Interaction and Intelligent Decision, Guangxi Academy Sciences, Nanning, China

    Changan Yuan

  4. School of Electronic and Information Engineering, Suzhou University of Science and Technology, Suzhou, 215009, China

    Hongjie Wu

  5. Institute of Science and Technology for Brain Inspired Intelligence (ISTBI), Fudan University, Shanghai, 200433, China

    Xingming Zhao

Authors
  1. Quanyi Pu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Changan Yuan
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Hongjie Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Xingming Zhao
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Quanyi Pu .

Editor information

Editors and Affiliations

  1. Tongji University, Shanghai, China

    De-Shuang Huang

  2. University of Ulsan, Ulsan, Korea (Republic of)

    Kang-Hyun Jo

  3. Xi'an Polytechnic University, Xi'an, China

    Junfeng Jing

  4. The University of Wollongong, North Wollongong, NSW, Australia

    Prashan Premaratne

  5. Polytecnic of Bari, Bari, Italy

    Vitoantonio Bevilacqua

  6. Liverpool John Moores University, Liverpool, UK

    Abir Hussain

Rights and permissions

Reprints and permissions

Copyright information

© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Pu, Q., Yuan, C., Wu, H., Zhao, X. (2022). Local Feature for Visible-Thermal PReID Based on Transformer. In: Huang, DS., Jo, KH., Jing, J., Premaratne, P., Bevilacqua, V., Hussain, A. (eds) Intelligent Computing Theories and Application. ICIC 2022. Lecture Notes in Computer Science, vol 13393. Springer, Cham. https://doi.org/10.1007/978-3-031-13870-6_29

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-13870-6_29

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-13869-0

  • Online ISBN: 978-3-031-13870-6

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation