Skip to main content

Advertisement

Springer Nature Link
Log in

Person re-identification based on human semantic parsing and message passing

  • Published:
The Journal of Supercomputing Aims and scope Submit manuscript

Abstract

As an effective method to assist intelligent management of public areas, person re-identification (re-ID) technology has been developed rapidly in recent years. However, there are still some insurmountable obstacles in practical applications, among which misalignment caused by some factors is a challenging problem. Unlike previous approaches that only superficially mine pedestrian information to solve this problem, we propose an aligned person re-ID method based on human semantic parsing and message passing. Our method achieves pixel-level alignment through the incorporation of semantic parsing and also utilizes the results of semantic parsing. It constructs a graph neural network based on the structure of the human body to achieve information interaction between various part features. Additionally, various semantic features and a global feature are considered and used in the loss function for the ensemble of features, thereby ensuring the discrimination and robustness. Such ensemble learning allows our method to perform well not only for the unaligned case, but also have the ability to handle occlusion. Thus, the proposed SPMP method achieves better performance than most existing methods on multiple popular datasets.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

Data Availability Statement

All data generated or analyzed during this study are included in this published article.

References

  1. Ye M, Shen J, Lin G, Xiang T, Shao L, Hoi SC (2021) Deep learning for person re-identification: a survey and outlook. IEEE Trans Pattern Anal Mach Intell 44:2872–2893

    Article  Google Scholar 

  2. Wei W, Yang W, Zuo E, Qian Y, Wang L (2022) Person re-identification based on deep learning-an overview. J Vis Commun Image Represent 82:103418

    Article  Google Scholar 

  3. Qiu J, Chai Y, Tian Z, Du X, Guizani M (2019) Automatic concept extraction based on semantic graphs from big data in smart city. IEEE Trans Comput Soc Syst 7(1):225–233

    Article  Google Scholar 

  4. Tang K, Miao D, Peng W, Wu J, Shi Y, Gu Z, Tian Z, Wang W (2021) Codes: Chamfer out-of-distribution examples against overconfidence issue. In: 2021 IEEE/CVF International Conference on Computer Vision (ICCV), pp 1133–1142. IEEE

  5. Ahmed E, Jones M, Marks TK (2015) An improved deep learning architecture for person re-identification. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 3908–3916

  6. Sun Y, Zheng L, Yang Y, Tian Q, Wang S (2018) Beyond part models: Person retrieval with refined part pooling (and a strong convolutional baseline). In: Proceedings of the European Conference on Computer Vision (ECCV), pp 480–496

  7. Zheng L, Shen L, Tian L, Wang S, Wang J, Tian Q (2015) Scalable person re-identification: A benchmark. In: Proceedings of the IEEE International Conference on Computer Vision, pp 1116–1124

  8. Ristani E, Solera F, Zou R, Cucchiara R, Tomasi C (2016) Performance measures and a data set for multi-target, multi-camera tracking. In: European Conference on Computer Vision, pp 17–35. Springer

  9. Zheng Z, Zheng L, Yang Y (2017) Unlabeled samples generated by gan improve the person re-identification baseline in vitro. In: Proceedings of the IEEE International Conference on Computer Vision, pp 3754–3762

  10. Zhao L, Li X, Zhuang Y, Wang J (2017) Deeply-learned part-aligned representations for person re-identification. In: Proceedings of the IEEE International Conference on Computer Vision, pp 3219–3228

  11. Fu Y, Wei Y, Zhou Y, Shi H, Huang G, Wang X, Yao Z, Huang T (2019) Horizontal pyramid matching for person re-identification. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol 33, pp 8295–8302

  12. Zheng F, Deng C, Sun X, Jiang X, Guo X, Yu Z, Huang F, Ji R (2019) Pyramidal person re-identification via multi-loss dynamic training. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp 8514–8522

  13. Luo H, Jiang W, Zhang X, Fan X, Qian J, Zhang C (2019) Alignedreid++: dynamically matching local information for person re-identification. Pattern Recogn 94:53–61

    Article  Google Scholar 

  14. Chen B, Deng W, Hu J (2019) Mixed high-order attention network for person re-identification. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp 371–381

  15. Li S, Bak S, Carr P, Wang X (2018) Diversity regularized spatiotemporal attention for video-based person re-identification. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 369–378

  16. Su C, Li J, Zhang S, Xing J, Gao W, Tian Q (2017) Pose-driven deep convolutional model for person re-identification. In: Proceedings of the IEEE International Conference on Computer Vision, pp 3960–3969

  17. Zhao H, Tian M, Sun S, Shao J, Yan J, Yi S, Wang X, Tang X (2017) Spindle net: Person re-identification with human body region guided feature decomposition and fusion. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 1077–1085

  18. Li W, Zhu X, Gong S (2018) Harmonious attention network for person re-identification. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 2285–2294

  19. Kalayeh MM, Basaran E, Gökmen M, Kamasak ME, Shah M (2018) Human semantic parsing for person re-identification. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 1062–1071

  20. Quispe R, Pedrini H (2019) Improved person re-identification based on saliency and semantic parsing with deep neural network models. Image Vis Comput 92:103809

    Article  Google Scholar 

  21. Zhu K, Guo H, Liu Z, Tang M, Wang J (2020) Identity-guided human semantic parsing for person re-identification. In: European Conference on Computer Vision, pp 346–363. Springer

  22. Moskvyak O, Maire F, Dayoub F, Baktashmotlagh M (2021) Keypoint-aligned embeddings for image retrieval and re-identification. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp 676–685

  23. Minaee S, Boykov YY, Porikli F, Plaza AJ, Kehtarnavaz N, Terzopoulos D (2021) Image segmentation using deep learning: A survey. IEEE Trans Pattern Anal Mach Intell. https://doi.org/10.1109/TPAMI.2021.3059968

    Article  Google Scholar 

  24. Liang X, Xu C, Shen X, Yang J, Liu S, Tang J, Lin L, Yan S (2015) Human parsing with contextualized convolutional neural network. In: Proceedings of the IEEE International Conference on Computer Vision, pp 1386–1394

  25. Xia F, Wang P, Chen L-C, Yuille AL (2016) Zoom better to see clearer: Human part segmentation with auto zoom net. In: ECCV, pp 648–663. Citeseer

  26. Chen L-C, Yang Y, Wang J, Xu W, Yuille AL (2016) Attention to scale: Scale-aware semantic image segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 3640–3649

  27. Liang X, Gong K, Shen X, Lin L (2018) Look into person: Joint body parsing & pose estimation network and a new benchmark. IEEE Trans Pattern Anal Mach Intell 41(4):871–885

    Article  Google Scholar 

  28. Ruan T, Liu T, Huang Z, Wei Y, Wei S, Zhao Y (2019) Devil in the details: Towards accurate single and multiple human parsing. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol 33, pp 4814–4821

  29. Li P, Xu Y, Wei Y, Yang Y (2020) Self-correction for human parsing. IEEE Trans Pattern Anal Mach Intell. https://doi.org/10.1109/TPAMI.2020.3048039

    Article  Google Scholar 

  30. Wang W, Zhang Z, Qi S, Shen J, Pang Y, Shao L (2019) Learning compositional neural information fusion for human parsing. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp 5703–5713

  31. Kipf TN, Welling M (2016) Semi-supervised classification with graph convolutional networks. arXiv preprint arXiv:1609.02907

  32. Xu B, Shen H, Cao Q, Qiu Y, Cheng X (2018) Graph wavelet neural network. In: International Conference on Learning Representations

  33. Hamilton W, Ying Z, Leskovec J (2017) Inductive representation learning on large graphs. Adv Neural Inf Process Syst 30

  34. Veličković P, Cucurull G, Casanova A, Romero A, Lio P, Bengio Y (2017) Graph attention networks. arXiv preprint arXiv:1710.10903

  35. Xu K, Hu W, Leskovec J, Jegelka S (2018) How powerful are graph neural networks? arXiv preprint arXiv:1810.00826

  36. Wei L, Zhang S, Yao H, Gao W, Tian Q (2018) GLAD: Global-local-alignment descriptor for scalable person re-identification. IEEE Trans Multimed 21(4):986–999

    Article  Google Scholar 

  37. Sun Y, Xu Q, Li Y, Zhang C, Li Y, Wang S, Sun J (2019) Perceive where to focus: Learning visibility-aware part-level features for partial person re-identification. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp 393–402

  38. Rahimpour A, Liu L, Taalimi A, Song Y, Qi H (2017) Person re-identification using visual attention. In: 2017 IEEE International Conference on Image Processing (ICIP), pp 4242–4246. IEEE

  39. Wang G, Yang S, Liu H, Wang Z, Yang Y, Wang S, Yu G, Zhou E, Sun J (2020) High-order information matters: Learning relation and topology for occluded person re-identification. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp 6449–6458

  40. Guo J, Yuan Y, Huang L, Zhang C, Yao J-G, Han K (2019) Beyond human parts: Dual part-aligned representations for person re-identification. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp 3642–3651

  41. He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 770–778

  42. Ioffe S, Szegedy C (2015) Batch normalization: Accelerating deep network training by reducing internal covariate shift. In: International Conference on Machine Learning, pp 448–456. PMLR

  43. Li W, Zhao R, Xiao T, Wang X (2004) Deepreid: Deep filter pairing neural network for person re-identification. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 152–159

  44. Zhong Z, Zheng L, Cao D, Li S (2017) Re-ranking person re-identification with k-reciprocal encoding. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 1318–1327

  45. Miao J, Wu Y, Liu P, Ding Y, Yang Y (2019) Pose-guided feature alignment for occluded person re-identification. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp 542–551

  46. Wang Y, Wang L, You Y, Zou X, Chen V, Li S, Huang G, Hariharan B, Weinberger KQ (2018) Resource aware person re-identification across multiple resolutions. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 8042–8051

  47. Zhong Z, Zheng L, Kang G, Li S, Yang Y (2020) Random erasing data augmentation. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol 34, pp 13001–13008

  48. Russakovsky O, Deng J, Su H, Krause J, Satheesh S, Ma S, Huang Z, Karpathy A, Khosla A, Bernstein M (2015) Imagenet large scale visual recognition challenge. Int J Comput Vis 115(3):211–252

    Article  MathSciNet  Google Scholar 

  49. Wang J, Zhu X, Gong S, Li W (2018) Transferable joint attribute-identity deep learning for unsupervised person re-identification. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 2275–2284

  50. Liu J, Zha Z-J, Chen D, Hong R, Wang M (2019) Adaptive transfer network for cross-domain person re-identification. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp 7202–7211

  51. Lin S, Li H, Li C-T, Kot AC (2018) Multi-task mid-level feature alignment network for unsupervised cross-dataset person re-identification. arXiv preprint arXiv:1807.01440

  52. Deng W, Zheng L, Ye Q, Kang G, Yang Y, Jiao J (2018) Image-image domain adaptation with preserved self-similarity and domain-dissimilarity for person re-identification. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 994–1003

  53. Zhong Z, Zheng L, Zheng Z, Li S, Yang Y (2018) Camera style adaptation for person re-identification. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 5157–5166

  54. Si J, Zhang H, Li C-G, Kuen J, Kong X, Kot AC, Wang G (2018) Dual attention matching network for context-aware feature sequence based person re-identification. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 5363–5372

  55. Li Z, Lv J, Chen Y, Yuan J (2021) Person re-identification with part prediction alignment. Comput Vis Image Underst 205:103172

    Article  Google Scholar 

  56. Wang C, Zhang Q, Huang C, Liu W, Wang X (2018) Mancs: A multi-task attentional network with curriculum sampling for person re-identification. In: Proceedings of the European Conference on Computer Vision (ECCV), pp 365–381

  57. Hou R, Ma B, Chang H, Gu X, Shan S, Chen X (2019) Interaction-and-aggregation network for person re-identification. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp 9317–9326

  58. Yang W, Huang H, Zhang Z, Chen X, Huang K, Zhang S (2019) Towards rich feature discovery with class activation maps augmentation for person re-identification. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp 1389–1398

  59. Jin H, Lai S, Zhao G, Qian X (2021) Hashing person re-id with self-distilling smooth relaxation. Neurocomputing 455:111–124

    Article  Google Scholar 

  60. Yang F, Yan K, Lu S, Jia H, Xie X, Gao W (2019) Attention driven person re-identification. Pattern Recogn 86:143–155

    Article  Google Scholar 

  61. Sun Y, Zheng L, Deng W, Wang S (2017) Svdnet for pedestrian retrieval. In: Proceedings of the IEEE International Conference on Computer Vision, pp 3800–3808

  62. Xu J, Zhao R, Zhu F, Wang H, Ouyang W (2018) Attention-aware compositional network for person re-identification. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 2119–2128

  63. Liu J, Ni B, Yan Y, Zhou P, Cheng S, Hu J (2018) Pose transferrable person re-identification. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 4099–4108

  64. Sarfraz MS, Schumann A, Eberle A, Stiefelhagen R (2018) A pose-sensitive embedding for person re-identification with expanded cross neighborhood re-ranking. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 420–429

  65. Qian X, Fu Y, Xiang T, Wang W, Qiu J, Wu Y, Jiang Y-G, Xue X (2018) Pose-normalized image generation for person re-identification. In: Proceedings of the European Conference on Computer Vision (ECCV), pp 650–667

  66. Liu C, Chang X, Shen Y-D (2020) Unity style transfer for person re-identification. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp 6887–6896

  67. Serbetci A, Akgul YS (2020) End-to-end training of CNN ensembles for person re-identification. Pattern Recogn 104:107319

    Article  Google Scholar 

  68. Liu M, Yan X, Wang C, Wang K (2021) Segmentation mask-guided person image generation. Appl Intell 51(2):1161–1176

    Article  Google Scholar 

  69. Wang C, Song L, Wang G, Zhang Q, Wang X (2020) Multi-scale multi-patch person re-identification with exclusivity regularized softmax. Neurocomputing 382:64–70

    Article  Google Scholar 

  70. Zhang T, Sun X, Li X, Yi Z (2021) Image generation and constrained two-stage feature fusion for person re-identification. Appl Intell 51(11):7679–7689

    Article  Google Scholar 

  71. Xu F, Ma B, Chang H, Shan S (2020) Isosceles constraints for person re-identification. IEEE Trans Image Process 29:8930–8943

    Article  MathSciNet  MATH  Google Scholar 

  72. Huang H, Li D, Zhang Z, Chen X, Huang K (2018) Adversarially occluded samples for person re-identification. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 5098–5107

  73. Ge Y, Li Z, Zhao H, Yin G, Yi S, Wang X, et al.: Fd-gan: Pose-guided feature distilling gan for robust person re-identification. Adv Neural Inf Process Syst 31 (2018)

  74. He L, Liang J, Li H, Sun Z (2018) Deep spatial feature reconstruction for partial person re-identification: Alignment-free approach. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 7073–7082

  75. He L, Sun Z, Zhu Y, Wang Y (2018) Recognizing partial biometric patterns. arXiv preprint arXiv:1810.07399

Download references

Acknowledgements

This work was supported by the National Natural Science Foundation of China (61573114) and the Science and Technology on Underwater Test and Control Laboratory under Grant (YS24071804).

Author information

Authors and Affiliations

  1. College of Intelligent Systems Science and Engineering, Harbin Engineering University, Harbin, 150001, China

    Chunyan Lyu, Tong Xu, Kejun Wang & Jing Chen

  2. School of Information Technology, Beijing Institute of Technology Zhuhai, Zhuhai, China

    Kejun Wang

Authors
  1. Chunyan Lyu
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Tong Xu
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Kejun Wang
    View author publications

    You can also search for this author inPubMed Google Scholar

  4. Jing Chen
    View author publications

    You can also search for this author inPubMed Google Scholar

Corresponding author

Correspondence to Kejun Wang.

Ethics declarations

Conflict of Interests

The authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Lyu, C., Xu, T., Wang, K. et al. Person re-identification based on human semantic parsing and message passing. J Supercomput 79, 5223–5247 (2023). https://doi.org/10.1007/s11227-022-04866-w

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11227-022-04866-w

Keywords