Skip to main content

Advertisement

Channel enhanced cross-modality relation network for visible-infrared person re-identification

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

Visible-infrared person re-identification (VI Re-ID) is designed to perform pedestrian retrieval on non-overlapping visible-infrared cameras, and it is widely employed in intelligent surveillance. For the VI Re-ID task, one of the main challenges is the huge modality discrepancy between the visible and infrared images. Therefore, mining more shared features in the cross-modality task turns into an important issue. To address this problem, this paper proposes a novel framework for feature learning and feature embedding in VI Re-ID, namely Channel Enhanced Cross-modality Relation Network (CECR-Net). More specifically, the network contains three key modules. In the first module, to shorten the distance between the original modalities, a channel selection operation is applied to the visible images, the robustness against color variations is improved by randomly generating three-channel R/G/B images. The module also exploits the low- and mid-level information of the visible and auxiliary modal images through a feature parameter-sharing strategy. Considering that the body sequences of pedestrians are not easy to change with modality, CECR-Net designs two modules based on relation network for VI Re-ID, namely the intra-relation learning and the cross-relation learning modules. These two modules help to capture the structural relationship between body parts, which is a modality-invariant information, disrupting the isolation between local features. Extensive experiments on the two public benchmarks indicate that CECR-Net is superior compared to the state-of-the-art methods. In particular, for the SYSU-MM01 dataset, the Rank1 and mAP reach 76.83% and 71.56% in the "All Search" mode, respectively.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

Data availability and access

The additional datasets generated during and analyzed during the current study are available from the corresponding author upon reasonable request.

References

  1. Zheng L, Yang Y, Hauptmann AG (2016) Person re-identification: Past, present and future. CoRR. abs/1610.02984

  2. Wu A, Zheng W-S, Yu H-X, Gong S, Lai J (2017) Rgb-infrared cross-modality person re-identification. In: 2017 IEEE International conference on computer vision (ICCV), pp 5390–5399

  3. Ye M, Shen J, Lin G, Xiang T, Shao L, Hoi SCH (2021) Deep learning for person re-identification: A survey and outlook. IEEE Transactions on pattern analysis and machine intelligence, pp 1–1

  4. Ye M, Wu Z, Chen C, Du B (2024) Channel augmentation for visible-infrared re-identification. IEEE Trans Pattern Anal Mach Intell 46(4):2299–2315

    Article  MATH  Google Scholar 

  5. Wei L, Zhang S, Yao H, Gao W, Tian Q (2019) Glad: Global local alignment descriptor for scalable person re-identification. IEEE Transactions multimedia 21(4):986–999

    Article  MATH  Google Scholar 

  6. Dai J, Zhang P, Wang D, Lu H, Wang H (2019) Video person re-identification by temporal residual learning. IEEE Trans Image Process 28(3):1366–1377

    Article  MathSciNet  MATH  Google Scholar 

  7. Ye M, Lan X, Li J, Yuen PC (2018) Hierarchical discriminative learning for visible thermal person re-identification. In: AAAI

  8. Liu H, Tan X, Zhou X (2020) Parameter sharing exploration and hetero-center triplet loss for visible-thermal person re-identification. IEEE Transactions on multimedia, pp 1–1

  9. Zhang Y, Wang H (2023) Diverse embedding expansion network and low-light cross-modality benchmark for visible-infrared person re-identification. In: 2023 IEEE/CVF Conference on computer vision and pattern recognition (CVPR), pp 2153–2162

  10. Kim S, Gwon S, Seo K (2024) Enhancing diverse intra-identity representation for visible-infrared person re-identification. In: 2024 IEEE/CVF Winter conference on applications of computer vision (WACV), pp 2501–2510

  11. Cheng Y, Xiao G, Tang X, Ma W, Gou X (2021) Two-phase feature fusion network for visible-infrared person re-identification. In: 2021 IEEE International conference on image processing (ICIP), pp 1149–1153

  12. Zhang Y, Kang Y, Zhao S, Shen J (2023) Dual-semantic consistency learning for visible-infrared person re-identification. IEEE Trans Inf Forensics Secur 18:1554–1565

    Article  MATH  Google Scholar 

  13. Wei Z, Yang X, Wang N, Gao X (2021) Flexible body partition-based adversarial learning for visible infrared person re-identification. IEEE Transactions on neural networks and learning systems, pp 1–12

  14. Liu Q, Teng Q, Chen H, Li B, Qing L (2022) Dual adaptive alignment and partitioning network for visible and infrared cross-modality person re-identification. Appl Intell 52:547–563

    Article  Google Scholar 

  15. Kim M, Kim S, Park J, Park S, Sohn K (2023) Partmix: Regularization strategy to learn part discovery for visible-infrared person re-identification. In: 2023 IEEE/CVF Conference on computer vision and pattern recognition (CVPR), pp 18621–18632

  16. Ye M, Lan X, Wang Z, Yuen PC (2020) Bi-directional center-constrained top-ranking for visible thermal person re-identification. IEEE Trans Inf Forensics Secur 15:407–419

    Article  MATH  Google Scholar 

  17. Zhu Y, Yang Z, Wang L, Zhao S, Hu X, Tao D (2020) Hetero-center loss for cross-modality person re-identification. Neurocomputing 386:97–109

    Article  MATH  Google Scholar 

  18. Ye H, Liu H, Meng F, Li X (2021) Bi-directional exponential angular triplet loss for rgb-infrared person re-identification. IEEE Trans Image Process 30:1583–1595

    Article  MATH  Google Scholar 

  19. Ye H, Liu H, Meng F, Li X (2021) Bi-directional exponential angular triplet loss for rgb-infrared person re-identification. IEEE Trans Image Process 30:1583–1595

    Article  MATH  Google Scholar 

  20. Zhong X, Lu T, Huang W, Ye M, Jia X, Lin C-W (2021) Grayscale enhancement colorization network for visible-infrared person re-identification. IEEE Transactions on circuits and systems for video technology, pp 1–1

  21. Zhang Y, Yan Y, Lu Y, Wang H (2021) Towards a unified middle modality learning for visible-infrared person re-identification. New York, NY, USA, pp 788–796

    MATH  Google Scholar 

  22. Tan X, Chai Y, Chen F, Liu H (2022) A fourier-based semantic augmentation for visible-thermal person re-identification. IEEE Signal Process Lett 29:1684–1688

    Article  MATH  Google Scholar 

  23. Liu J, Song W, Chen C, Liu F (2021) Cross-modality person re-identification via channel-based partition network. Appl Intell (8)

  24. Li D, Wei X, Hong X, Gong Y (2020) Infrared-visible cross-modal person re-identification with an x modality. In: Proceedings of the AAAI conference on artificial intelligence, pp 4610–4617

  25. Liu H, Xia D, Jiang W (2023) Towards homogeneous modality learning and multi-granularity information exploration for visible-infrared person re-identification. IEEE Journal of selected topics in signal processing, 17(3):545–559

    Article  MATH  Google Scholar 

  26. Varior RR, Shuai B, Lu J, Xu D, Wang G (2016) A siamese long short-term memory architecture for human re-identification. In: Computer vision – ECCV 2016, Springer, Cham, pp 135–153

  27. Park H, Ham B (2020) Relation network for person re-identification. In: 2020 AAAI Conference on artificial intelligence

  28. Zhang Z, Lan C, Zeng W, Jin X, Chen Z (2020) Relation-aware global attention for person re-identification. In: 2020 IEEE/CVF Conference on computer vision and pattern recognition (CVPR), pp 3183–3192

  29. Liao S, Hu Y, Zhu X et al (2015) Person re-identification by local maximal occurrence representation and metric learning. In: IEEE Conference on computer vision and pattern recognition, Boston, USA, pp 2197–2206

  30. You J, Wu A, Li X, Zheng WS (2016) Top-push video-based person re-identification. In: IEEE Conference on computer vision and pattern recognition., Las Vegas, USA, pp 1345–1353

  31. Yao H, Zhang S, Hong R, Zhang Y, Xu C, Tian Q (2019) Deep representation learning with part loss for person re-identification. IEEE Trans Image Process 28(6):2860–2871

    Article  MathSciNet  MATH  Google Scholar 

  32. Liu H, Cheng J, Wang W, Su Y, Bai H (2020) Enhancing the discriminative feature learning for visible-thermal cross-modality person re-identification. Neurocomputing 398:11–19

    Article  MATH  Google Scholar 

  33. Yin J, Ma Z, Xie J, Nie S, Liang K, Guo J (2022) Dual-granularity feature alignment for cross-modality person re-identification. Neurocomputing 511:78–90

    Article  Google Scholar 

  34. Zhang L, Du G, Liu F, Tu H, Shu X (2021) Global-local multiple granularity learning for cross-modality visible-infrared person reidentification. IEEE Transactions on neural networks and learning systems, pp 1–11

  35. Feng J, Wu A, Zheng W-S (2023) Shape-erased feature learning for visible-infrared person re-identification. In: 2023 IEEE/CVF Conference on computer vision and pattern recognition (CVPR), pp 22752–22761

  36. Ye M, Wang Z, Lan X, Yuen PC (2018) Visible thermal person re-identification via dual-constrained top-ranking. In: Proceedings of the twenty-seventh international joint conference on artificial intelligence, IJCAI-18, pp 1092–1099

  37. Wen Y, Zhang K, Li Z, Qiao Y (2016) A discriminative feature learning approach for deep face recognition. In: Leibe B, Matas J, Sebe N, Welling M (eds) Computer Vision – ECCV 2016, Cham, pp 499–515

  38. Liu H, Chai Y, Tan X, Li D, Zhou X (2021) Strong but simple baseline with dual-granularity triplet loss for visible-thermal person re-identification. IEEE Signal Process Lett 28:653–657

    Article  MATH  Google Scholar 

  39. Dai P, Ji R, Wang H, Wu Q, Huang Y (2018) Cross-modality person re-identification with generative adversarial training. In: Proceedings of the twenty-seventh international joint conference on artificial intelligence, pp 677–683

  40. Wang Z, Wang Z, Zheng Y, Chuang Y-Y, Satoh S (2019) Learning to reduce dual-level discrepancy for infrared-visible person re-identification. In: 2019 IEEE/CVF Conference on computer vision and pattern recognition (CVPR), pp 618–626

  41. Wang G, Zhang T, Cheng J, Liu S, Yang Y, Hou Z (2019) Rgb-infrared cross-modality person re-identification via joint pixel and feature alignment. In: 2019 IEEE/CVF International conference on computer vision (ICCV), pp 3622–3631

  42. Wang GA, Yang T, Cheng J, Chang J, Liang X, Hou Z (2020) Cross-modality paired-images generation for rgb-infrared person re-identification. Proceedings of the AAAI conference on artificial intelligence

  43. Ye M, Ruan W, Du B, Shou MZ (2021) Channel augmented joint learning for visible-infrared recognition. In: 2021 IEEE/CVF International conference on computer vision (ICCV), pp 13547–13556. https://doi.org/10.1109/ICCV48922.2021.01331

  44. Liu H, Xia D, Jiang W (2023) Towards homogeneous modality learning and multi-granularity information exploration for visible-infrared person re-identification. IEEE Journal of Selected Topics in Signal Processing. 17(3):545–559

    Article  MATH  Google Scholar 

  45. Cui Z, Zhou J, Peng Y (2024) Dma: Dual modality-aware alignment for visible-infrared person re-identification. IEEE Trans Inf Forensics Secur 19:2696–2708

    Article  MATH  Google Scholar 

  46. Sun Y, Zheng L, Yang Y, Tian Q, Wang S (2018) Beyond part models: Person retrieval with refined part pooling (and a strong convolutional baseline). In: Ferrari V, Hebert M, Sminchisescu C, Weiss Y (eds) Computer Vision - ECCV 2018. Springer, Cham, pp 501–518

    Chapter  Google Scholar 

  47. Dat N, Hyung H, Ki K, Kang P (2017) Person recognition system based on a combination of body images from visible light and thermal cameras. Sensors. 17(3):605

    Article  MATH  Google Scholar 

  48. Hao Y, Wang N, Li J, Gao X (2019) Hsme: Hypersphere manifold embedding for visible thermal person re-identification. In: Proceedings of the AAAI conference on artificial intelligence, pp 8385–8392

  49. Ye M, Lan X, Wang Z, Yuen PC (2020) Bi-directional center-constrained top-ranking for visible thermal person re-identification. IEEE Trans Inf Forensics Secur 15:407–419

  50. Ye M, Shen J, Shao L (2021) Visible-infrared person re-identification via homogeneous augmented tri-modal learning. IEEE Trans Inf Forensics Secur 16:728–739

    Article  Google Scholar 

  51. Zhao J, Wang H, Zhou Y, Yao R, Chen S, El Saddik A (2022) Spatial-channel enhanced transformer for visible-infrared person re-identification. IEEE Transactions on multimedia, pp 1–1. https://doi.org/10.1109/TMM.2022.3163847

  52. Liu H, Ma S, Xia D, Li S (2023) Sfanet: A spectrum-aware feature augmentation network for visible-infrared person reidentification. IEEE Transactions on neural networks and learning systems 34(4):1958–1971

    Article  MATH  Google Scholar 

  53. Gwon S, Kim S, Seo K (2024) Balanced and essential modality-specific and modality-shared representations for visible-infrared person re-identification. IEEE Signal Process Lett 31:491–495

    Article  MATH  Google Scholar 

  54. Lu Y, Wu Y, Liu B, Zhang T, Li B, Chu Q, Yu N (2020) Cross-modality person re-identification with shared-specific feature transfer. In: 2020 IEEE/CVF Conference on computer vision and pattern recognition (CVPR), pp 13376–13386

Download references

Acknowledgements

This work was supported in part by National Natural Science Foundation of China under Grant 62177029 and 61807020, and in part by the Startup Foundation for Introducing Talent of Nanjing University of Posts and Communications under Grant NY221041 and NY222034, and in part by General Project of The Natural Science Foundation of Jiangsu Higher Education Institution of China 22KJB520025.

Author information

Authors and Affiliations

Authors

Contributions

All authors contributed to the study conception and design. Conceptualization: Wanru Song; Methodology: Wanru Song; Writing - original draft preparation: Wanru Song, Xinyi Wang; Writing - review and editing: Weimin Wu,Yuan Zhang; Funding acquisition: Feng Liu. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Wanru Song.

Ethics declarations

Ethical and informed consent for data used

The data covered in this manuscript comes from publicly available datasets, and there are no ethical implications to discuss. All datasets involved in the current study are listed in the section of Experiments of the manuscript. Data used in the experiments, such as the Rank accuracy rate, has been cited by the manuscript. This manuscript is published with the informed consent of all authors.

Competing Interests

All authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Song, W., Wang, X., Wu, W. et al. Channel enhanced cross-modality relation network for visible-infrared person re-identification. Appl Intell 55, 4 (2025). https://doi.org/10.1007/s10489-024-06057-x

Download citation

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s10489-024-06057-x

Keywords