Abstract
Person search aims to simultaneously localize and identify a query person from realistic and uncropped images, which consists of person detection and re-identification (Re-ID). Existing person search methods and datasets predominantly focus on the visible light domain, and have difficulty in alleviating modality discrepancies. Furthermore, existing visible-infrared person Re-ID methods struggle to adequately address occlusions and handle background interference effectively. To address the above issues simultaneously, we first construct a new large-scale dataset, Multi-Modality Person Search (MMPS), which tackles the lack of suitable benchmarks for person search in the visible-infrared domain. Encompassing challenges of complex background interferences and occlusions under modality discrepancies, MMPS includes 21,470 images and 1,012 identities across six different cameras. Furthermore, we propose a novel visible-infrared person search method that integrates detection and Re-ID into a progressive process. Specifically, Progressive Inclusion (PI) is proposed to explore backgrounds and provide adaptive proposals. To better tackle the complex occlusions under significant modality discrepancies, we present Discriminative Mix (DM) to synthesize more diverse samples, leveraging specific pattern map embedding. This strategy ensures that our model is not overfitted to specific patterns and is capable of identifying diverse and distinctive human parts. Extensive experiments demonstrate that our method (PI-DM) achieves state-of-the-art performance on the task of visible-infrared person search. Our dataset has been released on https://github.com/sysuchx/MMPS.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Cao, J., et al.: PSTR: end-to-end one-step person search with transformers. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 9458–9467 (2022)
Chai, Z., Ling, Y., Luo, Z., Lin, D., Jiang, M., Li, S.: Dual-stream transformer with distribution alignment for visible-infrared person re-identification. IEEE Trans. Circuits Syst. Video Technol. (2023)
Chen, D., Zhang, S., Ouyang, W., Yang, J., Tai, Y.: Person search via a mask-guided two-stream CNN model. In: Proceedings of the European Conference on Computer Vision, pp. 734–750 (2018)
Chen, D., Zhang, S., Yang, J., Schiele, B.: Norm-aware embedding for efficient person search. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 12615–12624 (2020)
Chen, H., Zhang, Q., Lai, J., Xie, X.: Unsupervised group re-identification via adaptive clustering-driven progressive learning. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 38, pp. 1054–1062 (2024)
Deng, J., Dong, W., Socher, R., Li, L.J., Li, K., Fei-Fei, L.: ImageNet: a large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255. IEEE (2009)
Dong, W., Zhang, Z., Song, C., Tan, T.: Instance guided proposal network for person search. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 2585–2594 (2020)
Han, C., et al.: DMRNet++: learning discriminative features with decoupled networks and enriched pairs for one-step person search. IEEE Trans. Pattern Anal. Mach. Intell. (2022)
He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)
Li, D., Wei, X., Hong, X., Gong, Y.: Infrared-visible cross-modal person re-identification with an X modality. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 34, pp. 4610–4617 (2020)
Li, Z., Miao, D.: Sequential end-to-end network for efficient person search. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 35, pp. 2011–2019 (2021)
Liang, T., Jin, Y., Liu, W., Li, Y.: Cross-modality transformer with modality mining for visible-infrared person re-identification. IEEE Trans. Multimed. (2023)
Lu, H., Zou, X., Zhang, P.: Learning progressive modality-shared transformers for effective visible-infrared person re-identification. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 37, pp. 1835–1843 (2023)
Paszke, A., et al.: Automatic differentiation in pytorch (2017)
Ren, S., He, K., Girshick, R., Sun, J.: Faster R-CNN: towards real-time object detection with region proposal networks. In: Advances in Neural Information Processing Systems, vol. 28 (2015)
Vaswani, A., et al.: Attention is all you need. In: Advances in Neural Information Processing Systems, vol. 30 (2017)
Wu, A., Zheng, W.S., Yu, H.X., Gong, S., Lai, J.: RGB-infrared cross-modality person re-identification. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 5380–5389 (2017)
Wu, Q., et al.: Discover cross-modality nuances for visible-infrared person re-identification. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4330–4339 (2021)
Xiao, T., Li, S., Wang, B., Lin, L., Wang, X.: Joint detection and identification feature learning for person search. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3415–3424 (2017)
Xiong, J., Lai, J.: Similarity metric learning for RGB-infrared group re-identification. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13662–13671 (2023)
Xu, S., et al.: PP-YOLOE: an evolved version of YOLO. arXiv preprint arXiv:2203.16250 (2022)
Yan, Y., et al.: Exploring visual context for weakly supervised person search. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 36, pp. 3027–3035 (2022)
Yan, Y., et al.: Anchor-free person search. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 7690–7699 (2021)
Ye, M., Ruan, W., Du, B., Shou, M.Z.: Channel augmented joint learning for visible-infrared recognition. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 13567–13576 (2021)
Yu, R., et al.: Cascade transformers for end-to-end person search. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 7267–7276 (2022)
Yu, X., Gong, Y., Jiang, N., Ye, Q., Han, Z.: Scale match for tiny person detection. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1257–1265 (2020)
Yun, S., Han, D., Oh, S.J., Chun, S., Choe, J., Yoo, Y.: CutMix: regularization strategy to train strong classifiers with localizable features. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 6023–6032 (2019)
Zhang, Q., Lai, C., Liu, J., Huang, N., Han, J.: FMCnet: feature-level modality compensation for visible-infrared person re-identification. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 7349–7358 (2022)
Zhang, Q., Lai, J., Feng, Z., Xie, X.: Seeing like a human: asynchronous learning with dynamic progressive refinement for person re-identification. IEEE Trans. Image Process. 31, 352–365 (2021)
Zhang, Q., Lai, J., Xie, X.: Learning modal-invariant angular metric by cyclic projection network for VIS-NIR person re-identification. IEEE Trans. Image Process. 30, 8019–8033 (2021)
Zhang, Q., Wang, L., Patel, V.M., Xie, X., Lai, J.: View-decoupled transformer for person re-identification under aerial-ground camera network. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 22000–22009 (2024)
Zheng, L., Zhang, H., Sun, S., Chandraker, M., Yang, Y., Tian, Q.: Person re-identification in the wild. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3346–3355 (2017)
Acknowledgements
This project was supported in part by the National Natural Science Foundation of China (62076258, U22A2095) and the Key-Area Research and Development Program of Guangzhou (202206030003).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2025 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Chen, H., Xiong, J., Huang, Y., Xie, X., Lai, JH. (2025). Visible-Infrared Person Search: A Novel Benchmark and Solution. In: Antonacopoulos, A., Chaudhuri, S., Chellappa, R., Liu, CL., Bhattacharya, S., Pal, U. (eds) Pattern Recognition. ICPR 2024. Lecture Notes in Computer Science, vol 15314. Springer, Cham. https://doi.org/10.1007/978-3-031-78341-8_5
Download citation
DOI: https://doi.org/10.1007/978-3-031-78341-8_5
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-78340-1
Online ISBN: 978-3-031-78341-8
eBook Packages: Computer ScienceComputer Science (R0)