Skip to main content
SpringerLink
Log in

RMHNet: A Relation-Aware Multi-granularity Hierarchical Network for Person Re-identification

  • Published:
Neural Processing Letters Aims and scope Submit manuscript

Abstract

Person re-identification (re-id) expects to find a picture according to a few clues that refer to the same pedestrian from an image database caught from different locations. Previous studies show that the exploration of local features may help for capturing more robust feature representations. However, part-level characteristics are demonstrated useful but easily confused with comparable attributes in corresponding locations. For this problem, we propose an effective relation-aware multi-granularity hierarchical network (RMHNet) to explore the value of correlation between part-level and global features. In this paper, with a concise structure, RMHNet conducts a uniform partitioning strategy to relocate images into horizontal and vertical stripes. We incorporate the relationship between part-level semantic representation with the entire image to mine structural relationship information of the probe image for potential critical identification clues. Extensive experiments demonstrated the remarkable advantage of our RMHNet to state-of-the-art methods on several public benchmarks includes 84.2% rank-1 accuracy on MSMT17 (Wei et al., in: Internaltional conference on computer vision and pattern recogintion (CVPR), pp 79–88, 2018. https://doi.org/10.1109/CVPR.2018.00016) datasets, which is currently the largest publicly available person re-id dataset

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

Notes

  1. All faces are masked for anonymization.

References

  1. Zhang S, Yang Y, Wang P, Liang G, Zhang X, Zhang Y (2021) Attend to the difference: cross-modality person re-identification via contrastive correlation. IEEE Trans Image Process 30:8861–8872. https://doi.org/10.1109/TIP.2021.3120881

    Article  Google Scholar 

  2. Zhai Y, Guo X, Lu Y, Li H (2019) In defense of the classification loss for person re-identification. In: International conference on computer vision and pattern recognition workshops (CVPRW), pp 1526–1535 . https://doi.org/10.1109/CVPRW.2019.00194

  3. Deng W, Zheng L, Sun Y, Jiao J (2021) Rethinking triplet loss for domain adaptation. IEEE Trans Circuits Syst Video Technol 31(1):29–37. https://doi.org/10.1109/TCSVT.2020.2968484

    Article  Google Scholar 

  4. Proença H, Yaghoubi E, Alirezazadeh P (2021) A quadruplet loss for enforcing semantically coherent embeddings in multi-output classification problems. IEEE Trans Inf Forensics Secur 16:800–811. https://doi.org/10.1109/TIFS.2020.3023304

    Article  Google Scholar 

  5. Qu F, Liu J, Liu X, Jiang L (2021) A multi-fault detection method with improved triplet loss based on hard sample mining. IEEE Trans Sustain Energy 12(1):127–137. https://doi.org/10.1109/TSTE.2020.2985217

    Article  Google Scholar 

  6. Pang J, Zhang D, Li H, Liu W, Yu Z (2021) Hazy re-id: an interference suppression model for domain adaptation person re-identification under inclement weather condition. In: IEEE international conference on multimedia and expo (ICME), pp 1–6. https://doi.org/10.1109/ICME51207.2021.9428462

  7. Bai Z, Wang Z, Wang J, Hu D, Ding E (2021) Unsupervised multi-source domain adaptation for person re-identification. In: International conference on computer vision and pattern recognition (CVPR), pp 12909–12918. https://doi.org/10.1109/CVPR46437.2021.01272

  8. Wang P, Zhao Z, Su F, Zu X, Boulgouris NV (2021) Horeid: deep high-order mapping enhances pose alignment for person re-identification. IEEE Trans Image Process 30:2908–2922. https://doi.org/10.1109/TIP.2021.3055952

    Article  MathSciNet  Google Scholar 

  9. Chen X, Fu C, Zhao Y, Zheng F, Song J, Ji R, Yang Y (2020) Salience-guided cascaded suppression network for person re-identification. In: International conference on computer vision and pattern recognition (CVPR), pp 3297–3307. https://doi.org/10.1109/CVPR42600.2020.00336

  10. Ji Z, Zou X, Lin X, Liu X, Huang T, Wu S (2020) An attention-driven two-stage clustering method for unsupervised person re-identification. In: European conference on computer vision (ECCV), vol 12373, pp 20–36. https://doi.org/10.1007/978-3-030-58604-1_2

  11. Gao S, Wang J, Lu H, Liu Z (2020) Pose-guided visible part matching for occluded person reid. In: International conference on computer vision and pattern recognition (CVPR), pp 11741–11749. https://doi.org/10.1109/CVPR42600.2020.01176

  12. Liu J, Zha Z.-J, Wu W, Zheng K, Sun Q (2021) Spatial-temporal correlation and topology learning for person re-identification in videos. In: International conference on computer vision and pattern recognition (CVPR), pp 4368–4377. https://doi.org/10.1109/CVPR46437.2021.00435

  13. Zhong S, Bao Z, Gong S, Xia K (2021) Person reidentification based on pose-invariant feature and B-KNN reranking. IEEE Trans Comput Soc Syst 8(5):1272–1281. https://doi.org/10.1109/TCSS.2021.3063318

    Article  Google Scholar 

  14. Shi L, Zhang Y, Cheng J, Lu H (2020) Skeleton-based action recognition with multi-stream adaptive graph convolutional networks. IEEE Trans Image Process 29:9532–9545. https://doi.org/10.1109/TIP.2020.3028207

    Article  MATH  Google Scholar 

  15. Peng W, Hong X (2020) Learning graph convolutional network for skeleton-based human action recognition by neural searching. In: Association for the advance of artificial intelligence (AAAI), vol 34, pp 2669–2676. https://doi.org/10.1609/aaai.v34i03.5652

  16. Zhong S, Wei M, Gong S, Xia K, Fu Y, Fu Q, Yin H (2021) Behavior prediction for unmanned driving based on dual fusions of feature and decision. IEEE Trans Intell Transp Syst 22(6):3687–3696. https://doi.org/10.1109/TITS.2020.3037926

    Article  Google Scholar 

  17. Li X, Makihara Y, Xu C, Yagi Y, Ren M (2020) Gait recognition via semi-supervised disentangled representation learning to identity and covariate features. In: International conference on computer vision and pattern recognition (CVPR), pp 13306–13316. https://doi.org/10.1109/CVPR42600.2020.01332

  18. Lian S, Jiang W, Hu H (2021) Attention-aligned network for person re-identification. IEEE Trans Circuits Syst Video Technol 31(8):3140–3153. https://doi.org/10.1109/TCSVT.2020.3037179

    Article  Google Scholar 

  19. Huang Y, Lian S, Hu H, Chen D, Su T (2021) Multiscale omnibearing attention networks for person re-identification. IEEE Trans Circuits Syst Video Technol 31(5):1790–1803. https://doi.org/10.1109/TCSVT.2020.3014167

    Article  Google Scholar 

  20. Wang K, Wang P, Ding C, Tao D (2021) Batch coherence-driven network for part-aware person re-identification. IEEE Trans Image Process 30:3405–3418. https://doi.org/10.1109/TIP.2021.3060909

    Article  Google Scholar 

  21. Sun Y, Zheng L, Li Y, Yang Y, Tian Q, Wang S (2021) Learning part-based convolutional features for person re-identification. IEEE Trans Pattern Anal Mach Intell 43(3):902–917. https://doi.org/10.1109/TPAMI.2019.2938523

    Article  Google Scholar 

  22. Yang X, Liu L, Wang N, Gao X (2021) A two-stream dynamic pyramid representation model for video-based person re-identification. IEEE Trans Image Process 30:6266–6276. https://doi.org/10.1109/TIP.2021.3093759

    Article  Google Scholar 

  23. Wei L, Zhang S, Gao W, Tian Q (2018) Person transfer gan to bridge domain gap for person re-identification. In: International conference on computer vision and pattern recognition (CVPR), pp 79–88.https://doi.org/10.1109/CVPR.2018.00016

  24. Zhang X, Yan Y, Xue J-H, Hua Y, Wang H (2021) Semantic-aware occlusion-robust network for occluded person re-identification. IEEE Trans Circuits Syst Video Technol 31(7):2764–2778. https://doi.org/10.1109/TCSVT.2020.3033165

    Article  Google Scholar 

  25. Siarohin A, Lathuilière S, Sangineto E, Sebe N (2021) Appearance and pose-conditioned human image generation using deformable gans. IEEE Trans Pattern Anal Mach Intell 43(4):1156–1171. https://doi.org/10.1109/TPAMI.2019.2947427

    Article  Google Scholar 

  26. Zhou Q, Fan H, Yang H, Su H, Zheng S, Wu S, Ling H (2021) Robust and efficient graph correspondence transfer for person re-identification. IEEE Trans Image Process 30:1623–1638. https://doi.org/10.1109/TIP.2019.2914575

    Article  Google Scholar 

  27. Liu Z, Zhang H, Chen Z, Wang Z, Ouyang W (2020) Disentangling and unifying graph convolutions for skeleton-based action recognition. In: International conference on computer vision and pattern recognition (CVPR), pp 140–149. https://doi.org/10.1109/CVPR42600.2020.00022

  28. Cheng K, Zhang Y (2020) Decoupling GCN with dropgraph module for skeleton-based action recognition. In: European conference on computer vision (ECCV), vol 12369, pp 536–553. https://doi.org/10.1007/978-3-030-58586-0_32

  29. Liu W, Fu S (2021) Human activity recognition by manifold regularization based dynamic graph convolutional networks. Neurocomputing 444:217–225. https://doi.org/10.1016/j.neucom.2019.12.150

    Article  Google Scholar 

  30. Chen G, Gu T, Lu J, Bao J-A, Zhou J (2021) Person re-identification via attention pyramid. IEEE Trans Image Process 30:7663–7676. https://doi.org/10.1109/TIP.2021.3107211

    Article  Google Scholar 

  31. Zhang Z, Lan C, Zeng W, Jin X, Chen Z (2020) Relation-aware global attention for person re-identification. In: International conference on computer vision and pattern recognition (CVPR), pp 3183–3192. https://doi.org/10.1109/CVPR42600.2020.00325

  32. Wang P, Zhao Z, Su F, Zhao Y, Wang H, Yang L, Li Y (2021) Deep multi-patch matching network for visible thermal person re-identification. IEEE Trans Multimed 23:1474–1488. https://doi.org/10.1109/TMM.2020.2999180

    Article  Google Scholar 

  33. Zheng Z, Zheng L, Yang Y (2018) A discriminatively learned CNN embedding for person reidentification. ACM Trans Multimed Comput Commun Appl 14(1):13–33. https://doi.org/10.1145/3159171

    Article  MathSciNet  Google Scholar 

  34. Lin Y, Zheng L, Zheng Z, Wu Y, Hu Z, Yan C, Yang Y (2019) Improving person re-identification by attribute and identity learning. Pattern Recogn 95:151–161. https://doi.org/10.1016/j.patcog.2019.06.006

    Article  Google Scholar 

  35. Ji H, Wang L, Zhou S, Tang W, Zheng N, Hua G (2021) Meta pairwise relationship distillation for unsupervised person re-identification. In: International conference on computer vision (ICCV), pp 3641–3650. https://doi.org/10.1109/ICCV48922.2021.00364

  36. Chong Y, Peng C (2021) Style transfer for unsupervised domain-adaptive person re-identification. Neurocomputing 422:314–321. https://doi.org/10.1016/j.neucom.2020.10.005

    Article  Google Scholar 

  37. Sun Y, Zheng L, Yang Y, Tian Q, Wang S (2018) Beyond part models: person retrieval with refined part pooling (and a strong convolutional baseline). In: European conference on computer vision (ECCV), vol 11208, pp 501–518. https://doi.org/10.1007/978-3-030-01225-0_30

  38. Zhao H, Tian M, Sun S, Shao J, Yan J, Yi S, Wang X, Tang X (2017) Spindle net: person re-identification with human body region guided feature decomposition and fusion. In: International conference on computer vision and pattern recognition (CVPR), pp 907–915. https://doi.org/10.1109/CVPR.2017.103

  39. Santoro A, Raposo D, Barrett DG, Malinowski M, Pascanu R, Battaglia P, Lillicrap T (2017) A simple neural network module for relational reasoning. In: International conference on neural information processing systems (NIPS)

  40. Park H, Ham B (2020) Relation network for person re-identification. In: Association for the advance of artificial intelligence (AAAI), vol 34, pp 11839–11847. https://doi.org/10.1609/aaai.v34i07.6857

  41. Wang G, Yang S, Liu H, Wang Z, Yang Y, Wang S, Yu G, Zhou E, Sun J (2020) High-order information matters: learning relation and topology for occluded person re-identification. In: International conference on computer vision and pattern recognition (CVPR), pp 6448–6457. https://doi.org/10.1109/CVPR42600.2020.00648

  42. He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: International conference on computer vision and pattern recognition (CVPR), pp 770–778. https://doi.org/10.1109/CVPR.2016.90

  43. Deng J, Dong W, Socher R, Li L-J, Li K, Fei-Fei L (2009) Imagenet: a large-scale hierarchical image database. In: International conference on computer vision and pattern Recognition (CVPR), pp. 248–255. https://doi.org/10.1109/CVPR.2009.5206848

  44. Zheng L, Huang Y, Lu H, Yang Y (2019) Pose-invariant embedding for deep person re-identification. IEEE Trans Image Process 28(9):4500–4509. https://doi.org/10.1109/TIP.2019.2910414

    Article  MathSciNet  MATH  Google Scholar 

  45. Miao J, Wu Y, Liu P, Ding Y, Yang Y (2019) Pose-guided feature alignment for occluded person re-identification. In: International conference on computer vision (ICCV), pp 542–551. https://doi.org/10.1109/ICCV.2019.00063

  46. Filip R, Giorgos T, Ondrej C (2017) Fine-tuning CNN image retrieval with no human annotation. IEEE Trans Pattern Anal Mach Intell 41:1655–1668. https://doi.org/10.1109/TPAMI.2018.2846566

    Article  Google Scholar 

  47. Hu J, Shen L, Albanie S, Sun G, Wu E (2020) Squeeze-and-excitation networks. IEEE Trans Pattern Anal Mach Intell 42(8):2011–2023. https://doi.org/10.1109/TPAMI.2019.2913372

    Article  Google Scholar 

  48. Zheng L, Shen L, Tian L, Wang S, Wang J, Tian Q (2015) Scalable person re-identification: a benchmark. In: International conference on computer vision (ICCV), pp 1116–1124. https://doi.org/10.1109/ICCV.2015.133

  49. Ristani E, Solera F, Zou R, Cucchiara R, Tomasi C (2016) Performance measures and a data set for multi-target, multi-camera tracking. In: European conference on computer vision (ECCV), vol 9914, pp 17–35. https://doi.org/10.1007/978-3-319-48881-3_2

  50. Li W, Zhao R, Xiao T, Wang X (2014) Deepreid: deep filter pairing neural network for person re-identification. In: International conference on computer vision and pattern recognition (CVPR), pp 152–159. https://doi.org/10.1109/CVPR.2014.27

  51. Song C, Huang Y, Ouyang W, Wang L (2018) Mask-guided contrastive attention model for person re-identification. In: International conference on computer vision and pattern recognition (CVPR), pp 1179–1188. https://doi.org/10.1109/CVPR.2018.00129

  52. Li W, Zhu X, Gong S (2018) Harmonious attention network for person re-identification. In: International conference on computer vision and pattern recognition (CVPR), pp 2285–2294. https://doi.org/10.1109/CVPR.2018.00243

  53. Wang C, Zhang Q, Huang C, Liu W, Wang X (2018) Mancs: a multi-task attentional network with curriculum sampling for person re-identification. In: European conference on computer vision (ECCV), vol 11208, pp 384–400. https://doi.org/10.1007/978-3-030-01225-0_23

  54. Chen B, Deng W, Hu J (2019) Mixed high-order attention network for person re-identification. In: International conference on computer vision (ICCV), pp 371–381. https://doi.org/10.1109/ICCV.2019.00046

  55. Fang P, Zhou J, Roy S, Petersson L, Harandi M (2019) Bilinear attention networks for person retrieval. In: International conference on computer vision (ICCV), pp 8029–8038. https://doi.org/10.1109/ICCV.2019.00812

  56. Ye M, Shen J, Lin G, Xiang T, Shao L, Hoi SCH (2021) Deep learning for person re-identification: a survey and outlook. IEEE Trans Pattern Anal Mach Intell Early Access. https://doi.org/10.1109/TPAMI.2021.3054775

    Article  Google Scholar 

  57. Fu Y, Wei Y, Zhou Y, Shi H, Huang G, Wang X, Yao Z, Huang T (2019) Horizontal pyramid matching for person re-identification. In: Association for the advance of artificial intelligence (AAAI), vol 33, pp 8295–8302. https://doi.org/10.1609/aaai.v33i01.33018295

  58. Wang G, Yuan Y, Chen X, Li J, Zhou X (2018) Learning discriminative features with multiple granularities for person re-identification. In: ACM international conference on multimedia (ACM), pp 274–282. https://doi.org/10.1145/3240508.3240552

  59. Zhang Z, Lan C, Zeng W, Chen Z (2019) Densely semantically aligned person re-identification. In: International conference on computer vision and pattern recognition (CVPR), pp 667–676. https://doi.org/10.1109/CVPR.2019.00076

  60. Zhou K, Yang Y, Cavallaro A, Xiang T (2019) Omni-scale feature learning for person re-identification. In: International conference on computer vision (ICCV), pp 3701–3711. https://doi.org/10.1109/ICCV.2019.00380

  61. Zhu K, Guo H, Liu Z, Tang M, Wang J (2020) Identity-guided human semantic parsing for person re-identification. In: European conference on computer vision (ECCV), vol 12348, pp 346–363. https://doi.org/10.1007/978-3-030-58580-8_21

  62. Zhang A, Gao Y, Niu Y, Liu W, Zhou Y (2021) Coarse-to-fine person re-identification with auxiliary-domain classification and second-order information bottleneck. In: International conference on computer vision and pattern recognition (CVPR), pp 598–608. https://doi.org/10.1109/CVPR46437.2021.00066

  63. Li Y, He J, Zhang T, Liu X, Zhang Y, Wu F (2021) Diverse part discovery: occluded person re-identification with part-aware transformer. In: International conference on computer vision and pattern recognition (CVPR), pp 2897–2906. https://doi.org/10.1109/CVPR46437.2021.00292

  64. Li H, Wu G, Zheng W-S (2021) Combined depth space based architecture search for person re-identification. In: International conference on computer vision and pattern recognition (CVPR), pp 6725–6734. https://doi.org/10.1109/CVPR46437.2021.00666

  65. Chen T, Ding S, Xie J, Yuan Y, Chen W, Yang Y, Ren Z, Wang Z (2019) Abd-net: attentive but diverse person re-identification. In: International conference on computer vision (ICCV), pp 8350–8360. https://doi.org/10.1109/ICCV.2019.00844

  66. Dai Z, Chen M, Gu X, Zhu S, Tan P (2019) Batch dropblock network for person re-identification and beyond. In: International conference on computer vision (ICCV), pp 3690–3700. https://doi.org/10.1109/ICCV.2019.00379

  67. Hou R, Ma B, Chang H, Gu X, Shan S, Chen X (2019) Interaction-and-aggregation network for person re-identification. In: International conference on computer vision and pattern recognition (CVPR), pp 9309–9318. https://doi.org/10.1109/CVPR.2019.00954

  68. Zheng Z, Yang X, Yu Z, Zheng L, Yang Y, Kautz J (2019) Joint discriminative and generative learning for person re-identification. In: International conference on computer vision and pattern recognition (CVPR), pp 2133–2142. https://doi.org/10.1109/CVPR.2019.00224

Download references

Acknowledgements

This work was supported by the New-Generation AI Major Scientific and Technological Special Project of Tianjin (18ZXZNGX00150) and the Special Foundation for Technology Innovation of Tianjin (21YDTPJC00250).

Author information

Authors and Affiliations

  1. School of Computer Science and Engineering, Tianjin University of Technology, Tianjin, 300384, China

    Gengsheng Xie & Xianbin Wen

  2. Key Laboratory of Computer Vision and Systems, Ministry of Education, Tianjin, 300384, China

    Gengsheng Xie & Xianbin Wen

  3. Tianjin Key Laboratory of Intelligence Computing and Novel Software Technology, Tianjin, 300384, China

    Gengsheng Xie & Xianbin Wen

Authors
  1. Gengsheng Xie
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xianbin Wen
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Xianbin Wen.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Xie, G., Wen, X. RMHNet: A Relation-Aware Multi-granularity Hierarchical Network for Person Re-identification. Neural Process Lett 55, 1433–1454 (2023). https://doi.org/10.1007/s11063-022-10946-y

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11063-022-10946-y

Keywords

Search

Navigation