Skip to main content
SpringerLink
Log in

A novel video-based pedestrian re-identification method of sequence feature distribution similarity measurement combined with deep learning

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

In this paper, we propose a novel video-sequence-based pedestrian re-identification method using the feature distribution similarity measurement between pedestrian video sequences (PRI-FDSM). We use the multiple granularity network combined with generative adversarial skew correction to extract and generate the feature point sets of the corrected pedestrian sequences. Then, we construct the corresponding probability function estimators for each pedestrian sequence using a radial basis function neural network to describe the feature distributions of specific sequences. Finally, we measure the similarity between the feature distributions of sequences to obtain re-identification results. The proposed method uses the distribution similarity measurement of the feature point sets of different sequences to make full use of all the image information of the specific pedestrian in a sequence. Thus, our method can mitigate the problem of insufficient use of the details of some images in a sequence, which commonly occurs in existing fusion feature point measurement methods. Besides, we correct the input skewed pedestrian sequences and achieve posture unification for the input sequences, which effectively mitigates the posture skewing problem of the photographed pedestrians in real-world surveillance scenes. We also build a dataset that more accurately represents the real-world surveillance scenes that contain pedestrian sequences with skewed postures. The results of the ablation experiment on iLIDS-VID and this dataset demonstrate the effectiveness of the proposed distribution-based similarity measurement method. We also compare the performance of the proposed method and several state-of-the-art methods on our dataset. Experimental results show that the indices of our method are all higher than those of the existing methods, and its mAP, Rank-1 and Rank-5 surpass the second best by 3.7%, 1.3% and 1.7% respectively.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13

Similar content being viewed by others

Data Availability

The datasets generated during and analysed during the current study are available in the iLIDS-VID repository, http://www.eecs.qmul.ac.uk/~xiatian/downloads_qmul_iLIDS-VID_ReID_dataset.html, and PRID2011 repository, https://www.tugraz.at/institute/icg/research/team-bischof/lrs/downloads/PRID11/. Our pedestrian sequence dataset generated during and analysed during the current study are available from the corresponding author on reasonable request.

References

  1. Wang G, Yuan Y, Chen X, Li J, Zhou X (2018) Learning discriminative features with multiple granularities for person re-identification. In: Proceedings of the 26th ACM international conference on multimedia, pp 274–282

  2. Ge Y, Gu X, Chen M, Wang H, Yang D (2018) Deep multi-metric learning for person re-identification. In: 2018 IEEE International conference on multimedia and Expo (ICME), pp 1–6. IEEE

  3. Sun Y, Zheng L, Yang Y, Tian Q, Wang S (2018) Beyond part models: Person retrieval with refined part pooling (and a strong convolutional baseline). In: Proceedings of the european conference on computer vision (ECCV), pp 480–496

  4. Hou R, Ma B, Chang H, Gu X, Shan S, Chen X (2019) Interaction-and-aggregation network for person re-identification. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 9317–9326

  5. Zheng M, Karanam S, Wu Z, Radke RJ (2019) Re-identification with consistent attentive siamese networks. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 5735–5744

  6. Su J, He X, Qing L, Cheng Y, Peng Y (2021) An enhanced siamese angular softmax network with dual joint-attention for person re-identification. Appl Intell, pp 1–19

  7. Li R, Zhang B, Teng Z, Fan J (2021) A divide-and-unite deep network for person re-identification. Appl Intell 51(3):1479–1491

    Article  Google Scholar 

  8. Zheng W-S, Hong J, Jiao J, Wu A, Zhu X, Gong S, Qin J, Lai J (2021) Joint bilateral-resolution identity modeling for cross-resolution person re-identification. Int J Comput Vis, pp 1–21

  9. Chong Y, Peng C, Zhang C, Wang Y, Feng W, Pan S (2021) Learning domain invariant and specific representation for cross-domain person re-identification. Appl Intell, pp 1–14

  10. Li W, Zhu X, Gong S (2018) Harmonious attention network for person re-identification. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 2285–2294

  11. Liu Y, Yan J, Ouyang W (2017) Quality aware network for set to set recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 5790–5799

  12. Gao J, Nevatia R (2018) Revisiting temporal modeling for video-based person reid. coRR

  13. Liu C-T, Wu C-W, Wang Y-CF, Chien S-Y (2019) Spatially and temporally efficient non-local attention network for video-based person re-identification. In: British machine vision conference

  14. Hou R, Chang H, Ma B, Shan S, Chen X (2020) Temporal complementary learning for video person re-identification. In: European conference on computer vision, pp 388–405. Springer

  15. Gu X, Chang H, Ma B, Zhang H, Chen X (2020) Appearance-preserving 3d convolution for video-based person re-identification. In: European conference on computer vision, pp 228–243. Springer

  16. Breckon TP, Alsehaim A (2021) Not 3d re-id: Simple single stream 2d convolution for robust video re-identification. In: 2020 25th International conference on pattern recognition (ICPR), pp 5190–5197. IEEE

  17. Yang X, Liu L, Wang N, Gao X (2021) A two-stream dynamic pyramid representation model for video-based person re-identification. IEEE Trans on Image Process 30:6266–6276

    Article  Google Scholar 

  18. Aich A, Zheng M, Karanam S, Chen T, Roy-Chowdhury AK, Wu Z (2021) Spatio-temporal representation factorization for video-based person re-identification. In: Proceedings of the IEEE/CVF international conference on computer vision, pp 152–162

  19. Li P, Pan P, Liu P, Xu M, Yang Y (2020) Hierarchical temporal modeling with mutual distance matching for video based person re-identification. IEEE Trans Circuits Syst Video Technol 31(2):503–511

    Article  Google Scholar 

  20. Liu J, Zha Z-J, Chen X, Wang Z, Zhang Y (2019) Dense 3d-convolutional neural network for person re-identification in videos. Trans Multimedia Comput Commun Appl (TOMM) 15(1s):1–19

    Google Scholar 

  21. Chen G, Rao Y, Lu J, Zhou J (2020) Temporal coherence or temporal motion: Which is more critical for video-based person re-identification?. In: European conference on computer vision, pp 660–676. Springer

  22. Song W, Zheng J, Wu Y, Chen C, Liu F (2021) Discriminative feature extraction for video person re-identification via multi-task network. Appl Intell 51(2):788–803

    Article  Google Scholar 

  23. McLaughlin N, Del Rincon JM, Miller P (2016) Recurrent convolutional network for video-based person re-identification. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1325–1334

  24. Liu H, Jie Z, Jayashree K, Qi M, Jiang J, Yan S, Feng J (2017) Video-based person re-identification with accumulative motion context. IEEE Trans Circuits Syst Video Technol 28(10):2788–2802

    Article  Google Scholar 

  25. Matsukawa T, Okabe T, Suzuki E, Sato Y (2019) Hierarchical gaussian descriptors with application to person re-identification. IEEE Trans Pattern Anal Mach Intell 42(9):2179–2194

    Article  Google Scholar 

  26. Yu T, Li D, Yang Y, Hospedales TM, Xiang T (2019) Robust person re-identification by modelling feature uncertainty. In: Proceedings of the IEEE/CVF international conference on computer vision, pp 552–561

  27. Wang J, Li Y, Miao Z (2017) Siamese cosine network embedding for person re-identification. In: CCF Chinese conference on computer vision, pp 52–362. Springer

  28. Hu B, Xu J, Wang X (2021) Learning generalizable deep feature using triplet-batch-center loss for person re-identification. Sci China Inf Sci 64:1–2

    Article  Google Scholar 

  29. Hu X, Wei D, Wang Z, Shen J, Ren H (2021) Hypergraph video pedestrian re-identification based on posture structure relationship and action constraints. Pattern Recognit 107688:111

    Google Scholar 

  30. Jaderberg M, Simonyan K, Zisserman A et al (2015) Spatial transformer networks. Adv Neural Inf Process Syst 28:2017–2025

    Google Scholar 

  31. Goodfellow I, Bengio Y, Courville A (2016) Deep Learning. MIT Press

  32. Wang T, Gong S, Zhu X, Wang S (2014) Person re-identification by video ranking. In: European conference on computer vision, pp 688–703. Springer

  33. Hirzer M, Beleznai C, Roth PM, Bischof H (2011) Person re-identification by descriptive and discriminative classification. In: Scandinavian conference on image analysis, pp 91–102. Springer

  34. Dai J, Zhang P, Wang D, Lu H, Wang H (2019) Video person re-identification by temporal residual learning. IEEE Trans Image Process 28(3):1366–1377

    Article  MathSciNet  Google Scholar 

  35. Zhang W, Yu X, He X (2018) Learning bidirectional temporal cues for video-based person re-identification. IEEE Trans Circuits Syst Video Technol 28(10):2768–2776

    Article  Google Scholar 

  36. Sun L, Jiang Z, Song H, Lu Q, Men A (2018) Semi-coupled dictionary learning with relaxation label space transformation for video-based person re-identification. IEEE Access 6:12587–12597

    Article  Google Scholar 

  37. Liu Z, Wang Y, Li A (2018) Hierarchical integration of rich features for video-based person re-identification. IEEE Trans Circuits Syst Video Technol 29(12):3646–3659

    Article  Google Scholar 

  38. Zhang W, Hu S, Liu K, Zha Z (2018) Learning compact appearance representation for video-based person re-identification. IEEE Trans Circuits Syst Video Technol 29(8):2442–2452

    Article  Google Scholar 

  39. Wu L, Wang Y, Gao J, Li X (2019) Where-and-when to look: Deep siamese attention networks for video-based person re-identification. IEEE Trans Multimedia 21(6):1412–1424

    Article  Google Scholar 

  40. Zhang W, He X, Lu W, Qiao H, Li Y (2019) Feature aggregation with reinforcement learning for video-based person re-identification. IEEE Trans Neural Netw Learn Syst 30(12):3847–3852

    Article  Google Scholar 

  41. Song W, Zheng J, Wu Y, Chen C, Liu F (2019) A two-stage attribute-constraint network for video-based person re-identification. IEEE Access 7:8508–8518

    Article  Google Scholar 

  42. Song W, Zheng J, Wu Y, Chen C, Liu F (2020) Video-based person re-identification using a novel feature extraction and fusion technique. Multimed Tools Appl 79(17):12471–12491

    Article  Google Scholar 

  43. Liu J, Sun C, Xu X, Xu B, Yu S (2019) A spatial and temporal features mixture model with body parts for video-based person re-identification. Appl Intell 49(9):3436–3446

    Article  Google Scholar 

  44. Subramaniam A, Nambiar A, Mittal A (2019) Co-segmentation inspired attention networks for video-based person re-identification. In: Proceedings of the IEEE/CVF international conference on computer vision, pp 562–572

  45. Ye M, Shen J, Lin G, Xiang T, Shao L, Hoi SC (2021) Deep learning for person re-identification: A survey and outlook. IEEE Trans Pattern Anal Mach Intell

  46. Wang X, Girshick R, Gupta A, He K (2018) Non-local neural networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 7794–7803

  47. Liu C-T, Chen J-C, Chen C-S, Chien S-Y (2021) Video-based person re-identification without bells and whistles. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 1491–1500

  48. Hou R, Chang H, Ma B, Huang R, Shan S (2021) Bicnet-tks: Learning efficient spatial-temporal representation for video person re-identification. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 2014–2023

Download references

Acknowledgements

This work was supported in part by the National Natural Science Foundation of China under Grant (62071303, 61871269), Guangdong Basic and Applied Basic Research Foundation (2019A1515011861), Shenzhen Science and Technology Projection (JCYJ20190808151615540), China Postdoctoral Science Foundation (2021M702275)

Author information

Authors and Affiliations

  1. College of Electronics and Information Engineering, Shenzhen University, Shenzhen, 518160, Guangdong, China

    Jihong Pei, Jichen Zhang & Ziyang Ni

  2. College of Computer Science and Software Engineering, Shenzhen University, Shenzhen, 518160, Guangdong, China

    Yang Zhao

Authors
  1. Jihong Pei
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jichen Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ziyang Ni
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yang Zhao
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yang Zhao.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Pei, J., Zhang, J., Ni, Z. et al. A novel video-based pedestrian re-identification method of sequence feature distribution similarity measurement combined with deep learning. Appl Intell 53, 9779–9798 (2023). https://doi.org/10.1007/s10489-022-04021-1

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-022-04021-1

Keywords

Search

Navigation