La Zhang
Abstract
Visible-infrared person re-identification (Re-ID) has received increasing research attention for its great practical value in night-time surveillance scenarios. Due to the large variations in person pose, viewpoint, and occlusion in the same modality, as well as the domain gap brought by heterogeneous modality, this hybrid modality person matching task is quite challenging. Different from the metric learning methods for visible person re-ID, which only pose similarity constraints on class level, an efficient metric learning approach for visible-infrared person Re-ID should take both the class-level and modality-level similarity constraints into full consideration to learn sufficiently discriminative and robust features. In this article, the hybrid modality is divided into two types, within modality and cross modality. We first fully explore the variations that hinder the ranking results of visible-infrared person re-ID and roughly summarize them into three types: within-modality variation, cross-modality modality-related variation, and cross-modality modality-unrelated variation. Then, we propose a comprehensive metric learning framework based on four kinds of paired-based similarity constraints to address all the variations within and cross modality. This framework focuses on both class-level and modality-level similarity relationships between person images. Furthermore, we demonstrate the compatibility of our framework with any paired-based loss functions by giving detailed implementation of combing it with triplet loss and contrastive loss separately. Finally, extensive experiments of our approach on SYSU-MM01 and RegDB demonstrate the effectiveness and superiority of our proposed metric learning framework for visible-infrared person Re-ID.
- [1] . 2020. Hetero-Center loss for cross-modality person Re-identification. Neurocomputing 386 (2020), 97–109.Google Scholar
Cross Ref
- [2] . 2017. Beyond triplet loss: A deep quadruplet network for person re-identification. In Proceedings of the Conference on Computer Vision and Pattern Recognition.Google ScholarCross Ref
- [3] . 2016. Person re-identification by multi-channel parts-based CNN with improved triplet loss function. In Computer Vision and Pattern Recognition.Google Scholar
- [4] . 2020. Hi-CMD: Hierarchical cross-modality disentanglement for visible-infrared person re-identification. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR’20). IEEE.Google ScholarCross Ref
- [5] . 2018. Cross-modality person re-identification with generative adversarial training. In Proceedings of the 27th International Joint Conference on Artificial Intelligence IJCAI-18. Google ScholarDigital Library
- [6] . 2005. Histograms of oriented gradients for human detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR’05). Google ScholarDigital Library
- [7] . 2017. Person recognition system based on a combination of body images from visible light and thermal cameras. Sensors 17, 3 (2017), 605.
DOI: DOI: https://doi.org/10.3390/s17030605Google ScholarCross Ref - [8] Weijian Deng, Liang Zheng, Guoliang Kang, Yi Yang, Qixiang Ye, and Jianbin Jiao. 2018. Image-image domain adaptation with preserved self-similarity and domain-dissimilarity for person re-identification. In IEEE/CVF Conference on Computer Vision and Pattern Recognition. IEEE.Google Scholar
- [9] Zhangxiang Feng, Jianhuang Lai, and Xiaohua Xie. 2019. Learning modality-specific representations for visible-infrared person re-identification. IEEE Transactions on Image Processing 29 (2019), 579–590.Google Scholar
- [10] Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Y. Bengio. 2014. Generative adversarial nets. In Advances in Neural Information Processing Systems (NIPS’14). 2672–2680. Google ScholarDigital Library
- [11] Wang Guan’An, Zhang Tianzhu, Cheng Jian, Liu Si, Yang Yang, and Hou Zengguang. 2020. RGB-infrared cross-modality person re-identification via joint pixel and feature alignment. In IEEE/CVF International Conference on Computer Vision (ICCV’19). IEEE.Google Scholar
- [12] . 2019. A survey on deep learning based person re-identification. Acta Automat. Sin. 45, 11 (2019), 2032–2049.
DOI: DOI: https://doi.org/10.16383/j.aas.c180154Google Scholar - [13] . 2019. HSME: Hypersphere manifold embedding for visible thermal person re-identification. Proc. AAAI Conf. Artif. Intell. 33 (2019), 8385–8392.
DOI: DOI: https://doi.org/10.1609/aaai.v33i01.33018385 Google ScholarCross Ref - [14] . 2017. In defense of the triplet loss for person re-identification. CoRR abs/1703.07737 (2017).
DOI: DOI: https://doi.org/1703.07737Google Scholar - [15] Yan Huang, Jingsong Xu, Qiang Wu, Zhedong Zheng, Zhaoxiang Zhang, and Jian Zhang. 2018. Multi-pseudo regularized label for generated data in person re-identification. IEEE Transactions on Image Processing PP (2018), 1–1. https://doi.org/10.1109/TIP.2018.2874715Google Scholar
- [16] . 2020. A similarity inference metric for RGB-infrared cross-modality person re-identification. In Proceedings of the International Joint Conferences on Artificial Intelligence Organization. Google ScholarDigital Library
- [17] . 2020. Infrared-visible cross-modal person re-identification with an X modality. Proc. AAAI Conf. Artif. Intell. 34, 4 (2020), 4610–4617.Google Scholar
- [18] Shengcai Liao, Yang Hu, Xiangyu Zhu, and Stan Z. Li. 2015. Person re-identification by Local Maximal Occurrence representation and metric learning. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR’15). IEEE.Google Scholar
- [19] . 2020. Enhancing the discriminative feature learning for visible-thermal cross-modality person re-identification. Neurocomputing 398 (2020), 11–19.Google ScholarCross Ref
- [20] . 2017. End-to-end comparative attention networks for person re-identification. IEEE Trans. Image Process. 26, 99 (2017), 3492–3506.Google ScholarDigital Library
- [21] . 2020. Cross-modality person re-identification with shared-specific feature transfer. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR’20). IEEE.Google ScholarCross Ref
- [22] Ye Mang, Shen Jianbing, Lin Gaojie, Xiang Tao, Shao Ling, and Steven C. H. Hoi. 2021. Deep learning for person re-identification: a survey and outlook. IEEE Transactions on Pattern Analysis and Machine Intelligence PP, 99 (2021), 1–1.Google Scholar
- [23] Ye Mang, Lan Xiangyuan, and Leng Qingming. 2019. Modality-aware Collaborative Learning for Visible Thermal Person Re-Identification. In 27th ACM International Conference. ACM. Google ScholarDigital Library
- [24] Xuelin Qian, Yanwei Fu, Wenxuan Wang, Tao Xiang, Yang Wu, Yu-Gang Jiang, and Xiangyang Xue. 2018. Pose-normalized image generation for person re-identification. In Proceedings of the 15th European Conference, Munich, Germany. Springer, Cham.Google Scholar
- [25] Ergys Ristani and Carlo Tomasi. 2018. Features for multi-target multi-camera tracking and re-identification. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 6036–6046.Google Scholar
- [26] Hailin Shi, Yang Yang, Xiangyu Zhu, Shengcai Liao, Zhen Lei, and Stan Z. Li. 2016. Embedding deep metric for person re-identification: A study against large variations. In European Conference on Computer Vision. Springer, Cham, 732–748.Google Scholar
- [27] . 2016. Improved deep metric learning with multi-class n-pair loss objective. In Advances in Neural Information Processing Systems, , , , , and (Eds.), Vol. 29. Curran Associates, Inc., 1857–1865. Google ScholarDigital Library
- [28] . 2016. Gated siamese convolutional neural network architecture for human re-identification. In Proceedings of the European Conference on Computer Vision.Google ScholarCross Ref
- [29] . 2016. A siamese long short-term memory architecture for human re-identification. In Proceedings of the European Conference on Computer Vision.Google ScholarCross Ref
- [30] Wang Guan’An, Zhang Tianzhu, Yang Yang, Cheng Jian, Chang Jianlong, Liang Xu, and Hou Zengguang. 2020. Cross-modality paired-images generation for RGB-infrared person re-identification. Proceedings of the AAAI Conference on Artificial Intelligence 34, 7 (2020), 12144–12151.Google Scholar
- [31] . 2018. Person re-identification with cascaded pairwise convolutions. In Proceedings of the IEEE Conference on Computer Vision and Pattern Reconigtion.Google ScholarCross Ref
- [32] Longhui Wei, Shiliang Zhang, Wen Gao, and Qi Tian. 2018. Person transfer gan to bridge domain gap for person re-identification. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 79–88.Google Scholar
- [33] . 2017. RGB-infrared cross-modality person re-identification. In Proceedings of the International Conference on Computer Vision (ICCV’17). IEEE, Los Alamitos, CA.Google ScholarCross Ref
- [34] . 2018. Hierarchical discriminative learning for visible thermal person re-identification. In Proceedings of the AAAI Conference on Artificial Intelligence. AAAI.Google ScholarCross Ref
- [35] . 2018. Visible thermal person re-identification via dual-constrained top-ranking. In Proceedings of the 27th International Joint Conference on Artificial Intelligence (IJCAI’18). Google ScholarDigital Library
- [36] Shizhou Zhang, Yifei Yang, Peng Wang, Xiuwei Zhang, and Yanning Zhang. 2021. Attend to the difference: Cross-modality person re-identification via contrastive correlation. IEEE Transactions on Image Processing 30 (2021), 8861–8872.Google Scholar
- [37] . 2020. HPILN: A feature learning framework for cross-modality person re-identification. IET Image Process. 13, 14 (2020), 2897–2904.
DOI: DOI: https://doi.org/10.1049/iet-ipr.2019.0699Google ScholarCross Ref - [38] . 2016. Person re-identification: Past, present and future.Google Scholar
- [39] . 2019. Learning to reduce dual-level discrepancy for infrared-visible person re-identification. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR’19).Google Scholar
- [40] . 2018. Camera style adaptation for person re-identification. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.Google ScholarCross Ref
Index Terms
- Hybrid Modality Metric Learning for Visible-Infrared Person Re-Identification
Recommendations
Class-Aware Modality Mix and Center-Guided Metric Learning for Visible-Thermal Person Re-Identification
MM '20: Proceedings of the 28th ACM International Conference on MultimediaVisible thermal person re-identification (VT-REID) is an important and challenging task in that 1) weak lighting environments are inevitably encountered in real-world settings and 2) the inter-modality discrepancy is serious. Most existing methods ...
Modality-aware Collaborative Learning for Visible Thermal Person Re-Identification
MM '19: Proceedings of the 27th ACM International Conference on MultimediaVisible thermal person re-identification (VT-ReID) is a cross-modality pedestrian retrieval problem, which automatically searches persons between day-time visible images and night-time thermal images. Despite the extensive progress in single-modality ...
Visible-Infrared Cross-Modal Person Re-identification based on Positive Feedback
MMAsia '21: Proceedings of the 3rd ACM International Conference on Multimedia in AsiaVisible-infrared person re-identification (VI-ReID) is undoubtedly a challenging cross-modality person retrieval task with increasing appreciation. Compared to traditional person ReID that focuses on person images in a single RGB mode, VI-ReID suffers ...
Comments