skip to main content
10.1145/3652583.3658037acmconferencesArticle/Chapter ViewAbstractPublication PagesicmrConference Proceedingsconference-collections
research-article

Multidimensional Semantic Disentanglement Network for Clothes-Changing Person Re-Identification

Published: 07 June 2024 Publication History

Abstract

This study focuses on the Clothes-Changing Person Re-Identification (CC-ReID) problem, aiming to achieve precise recognition of the same pedestrian despite changes in attire. Despite some progress in this field, challenges persist in maintaining pedestrian identity consistency due to variations in clothing, leading to recognition disturbances. To address this, we propose a novel Multidimensional Semantic Disentanglement Network (MSD-Net). This network enhances the recognition capability for non-clothing areas by reducing reliance on clothing features and integrating discriminative and global features. Specifically, we employ semantic segmentation maps for pedestrian feature disentanglement, combined with RGB images, to effectively erase clothing features and consequently enhance focus on non-clothing areas. Additionally, we introduce a method to convert pedestrian semantic segmentation maps into dual-precision feature maps, utilizing a spatial attention mechanism to proactively learn distinctive pedestrian features, thereby further improving model performance. Extensive experiments on two standard CC-ReID datasets validate the effectiveness of our approach, outperforming existing state-of-the-art solutions. On the PRCC and VC-Clothes datasets, our model achieves Top-1 accuracies of 65.3% and 84.1%, respectively, in clothes-changing scenarios.

References

[1]
Jiaxing Chen, Xinyang Jiang, Fudong Wang, Jun Zhang, Feng Zheng, Xing Sun, and Wei-Shi Zheng. 2021a. Learning 3d shape feature for texture-insensitive person re-identification. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 8146--8155.
[2]
Xiumei Chen, Xiangtao Zheng, and Xiaoqiang Lu. 2021b. Bidirectional interaction network for person re-identification. IEEE Transactions on Image Processing, Vol. 30 (2021), 1935--1948.
[3]
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. 2009. Imagenet: A large-scale hierarchical image database. In 2009 IEEE conference on computer vision and pattern recognition. Ieee, 248--255.
[4]
Yongkang Ding, Rui Mao, Guodong Du, and Liyan Zhang. 2024 a. Clothes-Eraser: clothing-aware controllable disentanglement for clothes-changing person re-identification. Signal, Image and Video Processing (2024), 1--12.
[5]
Yongkang Ding, Yinghao Wu, Anqi Wang, Tiantian Gong, and Liyan Zhang. 2024 b. Disentangled body features for clothing change person re-identification. Multimedia Tools and Applications (2024), 1--22.
[6]
Chanho Eom, Wonkyung Lee, Geon Lee, and Bumsub Ham. 2021. Disentangled representations for short-term and long-term person re-identification. IEEE transactions on pattern analysis and machine intelligence, Vol. 44, 12 (2021), 8975--8991.
[7]
Ke Gong, Xiaodan Liang, Dongyu Zhang, Xiaohui Shen, and Liang Lin. 2017. Look into person: Self-supervised structure-sensitive learning and a new benchmark for human parsing. In Proceedings of the IEEE conference on computer vision and pattern recognition. 932--940.
[8]
Jianyang Gu, Kai Wang, Hao Luo, Chen Chen, Wei Jiang, Yuqiang Fang, Shanghang Zhang, Yang You, and Jian Zhao. 2023. MSINet: Twins Contrastive Search of Multi-Scale Interaction for Object ReID. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 19243--19253.
[9]
Xinqian Gu, Hong Chang, Bingpeng Ma, Shutao Bai, Shiguang Shan, and Xilin Chen. 2022. Clothes-changing person re-identification with RGB modality only. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 1060--1069.
[10]
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition. 770--778.
[11]
Shuting He, Hao Luo, Pichao Wang, Fan Wang, Hao Li, and Wei Jiang. 2021. Transreid: Transformer-based object re-identification. In Proceedings of the IEEE/CVF international conference on computer vision. 15013--15022.
[12]
Alexander Hermans, Lucas Beyer, and Bastian Leibe. 2017. In defense of the triplet loss for person re-identification. arXiv preprint arXiv:1703.07737 (2017).
[13]
Peixian Hong, Tao Wu, Ancong Wu, Xintong Han, and Wei-Shi Zheng. 2021. Fine-grained shape-appearance mutual learning for cloth-changing person re-identification. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 10513--10522.
[14]
Ruibing Hou, Bingpeng Ma, Hong Chang, Xinqian Gu, Shiguang Shan, and Xilin Chen. 2019. Interaction-And-Aggregation Network for Person Re-Identification. In 2019 IEEE /CVF Conference on Computer Vision and Pattern Recognition (CVPR ). IEEE, Long Beach, CA, USA, 9309--9318. https://doi.org/10.1109/CVPR.2019.00954
[15]
Yan Huang, Jingsong Xu, Qiang Wu, Yi Zhong, Peng Zhang, and Zhaoxiang Zhang. 2019. Beyond scalar neuron: Adopting vector-neuron capsules for long-term person re-identification. IEEE Transactions on Circuits and Systems for Video Technology, Vol. 30, 10 (2019), 3459--3471.
[16]
Xin Jin, Tianyu He, Kecheng Zheng, Zhiheng Yin, Xu Shen, Zhen Huang, Ruoyu Feng, Jianqiang Huang, Zhibo Chen, and Xian-Sheng Hua. 2022. Cloth-changing person re-identification from a single image with gait prediction and regularization. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 14278--14287.
[17]
Srikrishna Karanam, Yang Li, and Richard J Radke. 2015. Person re-identification with discriminatively trained viewpoint invariant dictionaries. In Proceedings of the IEEE international conference on computer vision. 4516--4524.
[18]
Diederik P Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014).
[19]
Peike Li, Yunqiu Xu, Yunchao Wei, and Yi Yang. 2020. Self-correction for human parsing. IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 44, 6 (2020), 3260--3271.
[20]
Yu-Jhe Li, Xinshuo Weng, and Kris M Kitani. 2021. Learning shape representations for person re-identification under clothing change. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision. 2432--2441.
[21]
Zechao Li, Jinhui Tang, and Tao Mei. 2018. Deep collaborative embedding for social image understanding. IEEE transactions on pattern analysis and machine intelligence, Vol. 41, 9 (2018), 2070--2083.
[22]
Yuxuan Liu, Hongwei Ge, Zhen Wang, Yaqing Hou, and Mingde Zhao. 2023. Clothes-Changing Person Re-identification via Universal Framework with Association and Forgetting Learning. IEEE Transactions on Multimedia (2023).
[23]
Xuelin Qian, Wenxuan Wang, Li Zhang, Fangrui Zhu, Yanwei Fu, Tao Xiang, Yu-Gang Jiang, and Xiangyang Xue. 2020. Long-term cloth-changing person re-identification. In Proceedings of the Asian Conference on Computer Vision.
[24]
Ramprasaath R Selvaraju, Michael Cogswell, Abhishek Das, Ramakrishna Vedantam, Devi Parikh, and Dhruv Batra. 2017. Grad-cam: Visual explanations from deep networks via gradient-based localization. In Proceedings of the IEEE international conference on computer vision. 618--626.
[25]
Xiujun Shu, Ge Li, Xiao Wang, Weijian Ruan, and Qi Tian. 2021a. Semantic-guided pixel sampling for cloth-changing person re-identification. IEEE Signal Processing Letters, Vol. 28 (2021), 1365--1369.
[26]
Xiujun Shu, Xiao Wang, Xianghao Zang, Shiliang Zhang, Yuanqi Chen, Ge Li, and Qi Tian. 2021b. Large-scale spatio-temporal person re-identification: Algorithms and benchmark. IEEE Transactions on Circuits and Systems for Video Technology, Vol. 32, 7 (2021), 4390--4403.
[27]
Xiaoxiao Sun and Liang Zheng. 2019. Dissecting person re-identification from the viewpoint of viewpoint. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 608--617.
[28]
Yifan Sun, Liang Zheng, Yi Yang, Qi Tian, and Shengjin Wang. 2018. Beyond part models: Person retrieval with refined part pooling (and a strong convolutional baseline). In Proceedings of the European conference on computer vision (ECCV). 480--496.
[29]
Jinhui Tang, Xiangbo Shu, Guo-Jun Qi, Zechao Li, Meng Wang, Shuicheng Yan, and Ramesh Jain. 2016. Tri-clustered tensor completion for social-aware image tag refinement. IEEE transactions on pattern analysis and machine intelligence, Vol. 39, 8 (2016), 1662--1674.
[30]
Fangbin Wan, Yang Wu, Xuelin Qian, Yixiong Chen, and Yanwei Fu. 2020. When person re-identification meets changing clothes. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops. 830--831.
[31]
Guan'an Wang, Shuo Yang, Huanyu Liu, Zhicheng Wang, Yang Yang, Shuliang Wang, Gang Yu, Erjin Zhou, and Jian Sun. 2020. High-order information matters: Learning relation and topology for occluded person re-identification. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 6449--6458.
[32]
Guanshuo Wang, Yufeng Yuan, Xiong Chen, Jiwei Li, and Xi Zhou. 2018. Learning discriminative features with multiple granularities for person re-identification. In Proceedings of the 26th ACM international conference on Multimedia. 274--282.
[33]
Zhikang Wang, Feng Zhu, Shixiang Tang, Rui Zhao, Lihuo He, and Jiangning Song. 2022. Feature erasing and diffusion network for occluded person re-identification. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 4754--4763.
[34]
Cheng Yan, Guansong Pang, Jile Jiao, Xiao Bai, Xuetao Feng, and Chunhua Shen. 2021. Occluded person re-identification with single-scale global representations. In Proceedings of the IEEE/CVF international conference on computer vision. 11875--11884.
[35]
Yuming Yan, Huimin Yu, Shuzhao Li, Zhaohui Lu, Jianfeng He, Haozhuo Zhang, and Runfa Wang. 2022. Weakening the Influence of Clothing: Universal Clothing Attribute Disentanglement for Person Re-Identification. In IJCAI. 1523--1529.
[36]
Mouxing Yang, Zhenyu Huang, Peng Hu, Taihao Li, Jiancheng Lv, and Xi Peng. 2022. Learning with twin noisy labels for visible-infrared person re-identification. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 14308--14317.
[37]
Qize Yang, Ancong Wu, and Wei-Shi Zheng. 2019. Person re-identification by contour sketch under moderate clothing change. IEEE transactions on pattern analysis and machine intelligence, Vol. 43, 6 (2019), 2029--2046.
[38]
Jianfu Zhang, Li Niu, and Liqing Zhang. 2020b. Person re-identification with reinforced attribute attention selection. IEEE Transactions on Image Processing, Vol. 30 (2020), 603--616.
[39]
Zhizheng Zhang, Cuiling Lan, Wenjun Zeng, Xin Jin, and Zhibo Chen. 2020a. Relation-aware global attention for person re-identification. In Proceedings of the ieee/cvf conference on computer vision and pattern recognition. 3186--3195.
[40]
Zhedong Zheng, Xiaohan Wang, Nenggan Zheng, and Yi Yang. 2022. Parameter-efficient person re-identification in the 3d space. IEEE Transactions on Neural Networks and Learning Systems (2022).
[41]
Zhedong Zheng, Xiaodong Yang, Zhiding Yu, Liang Zheng, Yi Yang, and Jan Kautz. 2019. Joint discriminative and generative learning for person re-identification. In proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2138--2147.
[42]
Zhun Zhong, Liang Zheng, Guoliang Kang, Shaozi Li, and Yi Yang. 2020. Random erasing data augmentation. In Proceedings of the AAAI conference on artificial intelligence, Vol. 34. 13001--13008.
[43]
Kuan Zhu, Haiyun Guo, Zhiwei Liu, Ming Tang, and Jinqiao Wang. 2020. Identity-guided human semantic parsing for person re-identification. In Computer Vision--ECCV 2020: 16th European Conference, Glasgow, UK, August 23--28, 2020, Proceedings, Part III 16. Springer, 346--363.

Cited By

View all

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
ICMR '24: Proceedings of the 2024 International Conference on Multimedia Retrieval
May 2024
1379 pages
ISBN:9798400706196
DOI:10.1145/3652583
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 07 June 2024

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. cloth-changing scenario
  2. computer vision
  3. multidimensional features
  4. person re-identification

Qualifiers

  • Research-article

Funding Sources

Conference

ICMR '24
Sponsor:

Acceptance Rates

Overall Acceptance Rate 254 of 830 submissions, 31%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)136
  • Downloads (Last 6 weeks)30
Reflects downloads up to 03 Mar 2025

Other Metrics

Citations

Cited By

View all
  • (2025)Swin-transformer for weak feature matchingScientific Reports10.1038/s41598-025-87309-915:1Online publication date: 23-Jan-2025
  • (2025)Unambiguous granularity distillation for asymmetric image retrievalNeural Networks10.1016/j.neunet.2025.107303(107303)Online publication date: Feb-2025
  • (2025)SFVE: visual information enhancement metaphor detection with multimodal splitting fusionThe Journal of Supercomputing10.1007/s11227-025-06958-981:3Online publication date: 7-Feb-2025
  • (2024)Text-Guided Prototype Generation for Occluded Person Re-IdentificationIEEE Signal Processing Letters10.1109/LSP.2024.345600731(2350-2354)Online publication date: 2024
  • (2024)Multi-person Pose Forecasting with Individual Interaction Perceptron and Prior LearningComputer Vision – ECCV 202410.1007/978-3-031-72649-1_23(402-419)Online publication date: 29-Sep-2024

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media