Skip to main content
Log in

POSNet: a hybrid deep learning model for efficient person re-identification

  • Published:
The Journal of Supercomputing Aims and scope Submit manuscript

Abstract

Person re-identification refers to the process of recognizing a person across several non-overlapping cameras. It is becoming increasingly important in computer vision for real-world surveillance applications. However, the deployment of person-re-identification systems as a surveillance system raises various challenges in their performance. These challenges include limited labeled data, occlusions conditions, human body postures, as well as inter- and intra-class variations. Such challenges deteriorate the effectiveness of person-re-identification systems and lead to the extraction of less discriminative features. Hence, to address these problems, we proposed a hybrid deep learning model, namely POSNet (pseudo-labeled omni-scale network) for efficient person re-identification. The proposed method is referred to as a hybrid because it combines label estimate with modified omni-scale feature learning, i.e., spatiotemporal-assisted omni-scale feature extraction to accomplish person re-identification. To further enhance omni-scale feature learning, we have proposed soft-pool-assisted attention mechanisms during spatial learning. More precisely, soft-pool preserves more important features, and that features are further emphasized by spatial and channel attention layers. Following on, this omni-scale with soft-pool attention learning extracts the spatial information from all frames of videos, and later on, the temporal learning is incorporated using the LSTM model. To handle limited labeled data problems, the proposed hybrid model first assigns pseudo-labels to the unlabeled data and adopts a progressive learning strategy to retrain the model on both labeled and unlabeled data with improved feature extraction, i.e., modified omni-scale feature learning. Moreover, the proposed POSNet model is validated on two large video-based person re-identification datasets, namely MARS and DukeMTMC-Video. It is observed from the research findings that the proposed POSNet outperformed the existing studies with the highest mAP and rank@1 score of 83.7 and 90.3%, respectively.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

Data availability

The data are publicly available and can also be provided on request to the corresponding author.

References

  1. Zheng L, Yang Y, Hauptmann AG (2016) Person re-identification: Past, present and future. arXiv preprint arXiv:1610.02984

  2. Bialkowski A, Denman S, Sridharan S, Fookes C, Lucey P (2012) A database for person re-identification in multi-camera surveillance networks. In: 2012 International Conference on Digital Image Computing Techniques and Applications (DICTA), pp 1-8

  3. Yadav A, Vishwakarma DK (2020) Person re-identification using deep learning networks: a systematic review. arXiv preprint arXiv:2012.13318

  4. Ye M, Shen J, Lin G, Xiang T, Shao L, Hoi SC (2021) Deep learning for person re-identification: a survey and outlook. IEEE Trans Pattern Anal Mach Intell 44:2872–2893

    Article  Google Scholar 

  5. Ahmed E, Jones M, Marks TK (2015) An improved deep learning architecture for person re-identification. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 3908–3916

  6. Wu D, Zheng S-J, Zhang X-P, Yuan C-A, Cheng F, Zhao Y et al (2019) Deep learning-based methods for person re-identification: a comprehensive review. Neurocomputing 337:354–371

    Article  Google Scholar 

  7. Li W, Zhao R, Xiao T, Wang X (2014) Deepreid: deep filter pairing neural network for person re-identification. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 152–159

  8. Cheng D, Gong Y, Zhou S, Wang J, Zheng N (2016) Person re-identification by multi-channel parts-based cnn with improved triplet loss function. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 1335–1344

  9. Tian Y, Li Q, Wang D, Wan B (2019) Robust joint learning network: improved deep representation learning for person re-identification. Multimed Tools Appl 78:24187–24203

    Article  Google Scholar 

  10. Sun Y, Zheng L, Yang Y, Tian Q, Wang S (2018) Beyond part models: person retrieval with refined part pooling (and a strong convolutional baseline). In: Proceedings of the European Conference on Computer Vision (ECCV), pp 480–496

  11. Xiao J, Li H, Qu G, Fujita H, Cao Y, Zhu J et al (2022) Hope: heatmap and offset for pose estimation. J Ambient Intell Humaniz Comput 13:2937–2949

    Article  Google Scholar 

  12. Zhao H, Tian M, Sun S, Shao J, Yan J, Yi S et al (2017) Spindle net: person re-identification with human body region guided feature decomposition and fusion. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 1077–1085

  13. Wei L, Zhang S, Yao H, Gao W, Tian Q (2017) Glad: global-local-alignment descriptor for pedestrian retrieval. In: Proceedings of the 25th ACM International Conference on Multimedia, pp 420–428

  14. Zhu K, Guo H, Liu Z, Tang M, Wang J (2020) Identity-guided human semantic parsing for person re-identification. In: European Conference on Computer Vision, pp 346–363

  15. Song C, Huang Y, Ouyang W, Wang L (2018) Mask-guided contrastive attention model for person re-identification. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 1179–1188

  16. Zhao L, Li X, Zhuang Y, Wang J (2017) Deeply-learned part-aligned representations for person re-identification. In: Proceedings of the IEEE International Conference on Computer Vision, pp 3219–3228

  17. Si J, Zhang H, Li C-G, Kuen J, Kong X, Kot AC et al (2018) Dual attention matching network for context-aware feature sequence based person re-identification. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 5363–5372

  18. Liu J, Ni B, Yan Y, Zhou P, Cheng S, Hu J (2018) Pose transferrable person re-identification. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 4099–4108

  19. Ma L, Sun Q, Georgoulis S, Van Gool L, Schiele B, Fritz M (2018) Disentangled person image generation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 99–108

  20. Qian X, Fu Y, Xiang T, Wang W, Qiu J, Wu Y et al (2018) Pose-normalized image generation for person re-identification. In: Proceedings of the European Conference on Computer Vision (ECCV), pp 650–667

  21. Ge Y, Li Z, Zhao H, Yin G, Yi S, Wang X (2018) Fd-gan: pose-guided feature distilling gan for robust person re-identification. In: Advances in Neural Information Processing Systems, vol 31

  22. Gray D, Brennan S, Tao H (2007) Evaluating appearance models for recognition, reacquisition, and tracking. In: Proceeding IEEE International Workshop on Performance Evaluation for Tracking and Surveillance (PETS), pp 1–7

  23. Ess A, Leibe B, Van Gool L (2007) Depth and appearance for mobile scene analysis. In: 2007 IEEE 11th International Conference on Computer Vision, pp 1–8

  24. Schwartz WR, Davis LS (2009) Learning discriminative appearance-based models using partial least squares. In: 2009 XXII Brazilian Symposium on Computer Graphics and Image Processing, pp 322–329

  25. Hirzer M, Beleznai C, Roth PM, Bischof H (2011) Person re-identification by descriptive and discriminative classification. In: Scandinavian Conference on Image analysis, pp 91–102

  26. Cheng DS, Cristani M, Stoppa M, Bazzani L, Murino V (2011) Custom pictorial structures for re-identification. Bmvc 1:6

    Google Scholar 

  27. Prosser BJ, Zheng W-S, Gong S, Xiang T, Mary Q (2010) Person re-identification by support vector ranking. Bmvc 2:6

    Google Scholar 

  28. Wang T, Gong S, Zhu X, Wang S (2014) Person re-identification by video ranking. In: European Conference on Computer Vision, pp 688–703

  29. Gevers T, Smeulders AW (1999) Color-based object recognition. Pattern Recognit 32:453–464

    Article  Google Scholar 

  30. Zheng L, Shen L, Tian L, Wang S, Wang J, Tian Q (2015) Scalable person re-identification: a benchmark. In Proceedings of the IEEE International Conference on Computer Vision, pp 1116–1124

  31. Dehghan A, Modiri Assari S, Shah M (2015) Gmmcp tracker: globally optimal generalized maximum multi clique problem for multiple object tracking. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 4091–4099

  32. Felzenszwalb P, Girshick R, Mcallester D, Ramanan D (2013) DPM & Latent SVM. Course Febr

  33. Ristani E, Solera F, Zou R, Cucchiara R, Tomasi C (2016) Performance measures and a data set for multi-target, multi-camera tracking. In: European Conference on Computer Vision, pp 17–35

  34. Wu Y, Lin Y, Dong X, Yan Y, Ouyang W, Yang Y (2018) Exploit the unknown gradually: one-shot video-based person re-identification by stepwise learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 5177–5186

  35. Gou M, Karanam S, Liu W, Camps O, Radke RJ (2017) DukeMTMC4ReID: a large-scale multi-camera person re-identification dataset. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp 10–19

  36. Zheng Z, Zheng L, Yang Y (2017) Unlabeled samples generated by gan improve the person re-identification baseline in vitro. In: Proceedings of the IEEE International Conference on Computer Vision, pp 3754–3762

  37. Gao S, Wang J, Lu H, Liu Z (2020) Pose-guided visible part matching for occluded person reid. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp 11744–11752

  38. Zheng W-S, Gong S, Xiang T (2011) Person re-identification by probabilistic relative distance comparison. CVPR 2011:649–656

    Google Scholar 

  39. Gray D, Tao H (2008) Viewpoint invariant pedestrian recognition with an ensemble of localized features. In European Conference on Computer Vision, pp 262–275

  40. Lin Y, Dong X, Zheng L, Yan Y, Yang Y (2019) A bottom-up clustering approach to unsupervised person re-identification. In Proceedings of the AAAI Conference on Artificial Intelligence, pp 8738–8745

  41. Zang X, Li G, Gao W, Shu X (2022) Exploiting robust unsupervised video person re-identification. IET Image Process 16:729–741

    Article  Google Scholar 

  42. Teng H, He T, Guo Y, Ding G (2020) A high-accuracy unsupervised person re-identification method using auxiliary information mined from datasets. arXiv preprint arXiv:2205.03124

  43. Gratacos B (2006) A robust algorithm and associated QC for finding the anisotropy directions for converted wave data. In: 68th EAGE Conference and Exhibition Incorporating SPE EUROPEC 2006, pp cp-2–00333

  44. Liu Z, Wang D, Lu H (2017) Stepwise metric promotion for unsupervised video person re-identification. In: Proceedings of the IEEE International Conference on Computer Vision, pp 2429–2438

  45. Ye M, Ma AJ, Zheng L, Li J, Yuen PC (2017) Dynamic label graph matching for unsupervised video re-identification. In: Proceedings of the IEEE International Conference on Computer Vision, pp 5142–5150

  46. Ma AJ, Li P (2015) Semi-supervised ranking for re-identification with few labeled image pairs. In: Asian Conference on Computer Vision, pp 598–613

  47. Xiang S, Fu Y, Guan M, Liu T (2022) Learning from self-discrepancy via multiple co-teaching for cross-domain person re-identification. Mach Learn. https://doi.org/10.1007/s10994-022-06184-x

    Article  Google Scholar 

  48. Ge Y, Chen D, Li H (2020) Mutual mean-teaching: Pseudo label refinery for unsupervised domain adaptation on person re-identification. arXiv preprint arXiv:2001.01526

  49. Xiang S, Fu Y, Liu T (2020) Progressive learning with style transfer for distant domain adaptation. IET Image Process 14:3527–3535

    Article  Google Scholar 

  50. Cho Y, Kim WJ, Hong S, Yoon S-E (2022) Part-based pseudo label refinement for unsupervised person re-identification. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp 7308–7318

  51. Nambiar A, Bernardino A, Nascimento JC (2019) Gait-based person re-identification: a survey. ACM Comput Surv (CSUR) 52:1–34

    Article  Google Scholar 

  52. Miao J, Wu Y, Yang Y (2021) Identifying visible parts via pose estimation for occluded person re-identification. IEEE Trans Neural Netw Learn Syst. https://doi.org/10.1109/TNNLS.2021.3059515

    Article  Google Scholar 

  53. Wang Y, Liao S, Shao L (2020) Surpassing real-world source training data: random 3D characters for generalizable person re-identification. In Proceedings of the 28th ACM International Conference on Multimedia, pp 3422–3430

  54. Almasawa MO, Elrefaei LA, Moria K (2019) A survey on deep learning-based person re-identification systems. IEEE Access 7:175228–175247

    Article  Google Scholar 

  55. Lin Y, Zheng L, Zheng Z, Wu Y, Hu Z, Yan C et al (2019) Improving person re-identification by attribute and identity learning. Pattern Recognit 95:151–161

    Article  Google Scholar 

  56. Schumann A, Stiefelhagen R (2017) Person re-identification by deep learning attribute-complementary information. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp 20–28

  57. Ma T, Yang M, Rong H, Qian Y, Tian Y, Al-Nabhan N (2021) Dual-path CNN with max gated block for text-based person re-identification. Image Vis Comput 111:104168

    Article  Google Scholar 

  58. Wang G, Fang Y, Wang J, Sun J (2016) Extensive comparison of visual features for person re-identification. In: Proceedings of the International Conference on Internet Multimedia Computing and Service, pp 192–196

  59. Tome P, Fierrez J, Vera-Rodriguez R, Nixon MS (2014) Soft biometrics and their application in person recognition at a distance. IEEE Trans Inf Forensics Secur 9:464–475

    Article  Google Scholar 

  60. Fan H, Zheng L, Yan C, Yang Y (2018) Unsupervised person re-identification: clustering and fine-tuning. ACM Trans Multimed Comput Commun Appl (TOMM) 14:1–18

    Article  Google Scholar 

  61. Stergiou A, Poppe R, Kalliatakis G (2019) Refining activation downsampling with SoftPool. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp 10357–10366

  62. Zhou K, Yang Y, Cavallaro A, Xiang T (2019) Omni-scale feature learning for person re-identification. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp 3702–3712

  63. Bak S, Corvee E, Bremond F, Thonnat M (2010) Person re-identification using spatial covariance regions of human body parts. In: 2010 7th IEEE International Conference on Advanced Video and Signal Based Surveillance, pp 435–440

  64. Imani Z, Soltanizadeh H (2019) Local binary pattern, local derivative pattern and skeleton features for RGB-D person re-identification. Natl Acad Sci Lett 42:233–238

    Article  Google Scholar 

  65. Wang S, Xu X, Liu L, Tian J (2020) Multi-level feature fusion model-based real-time person re-identification for forensics. J Real-Time Image Proc 17:73–81

    Article  Google Scholar 

  66. Fayyaz M, Yasmin M, Sharif M, Shah JH, Raza M, Iqbal T (2020) Person re-identification with features-based clustering and deep features. Neural Comput Appl 32:10519–10540

    Article  Google Scholar 

  67. Bukhari M, Bajwa KB, Gillani S, Maqsood M, Durrani MY, Mehmood I et al (2020) An efficient gait recognition method for known and unknown covariate conditions. IEEE Access 9:6465–6477

    Article  Google Scholar 

  68. Yasmin S, Durrani MY, Gillani S, Bukhari M, Maqsood M, Zghaibeh M (2022) Small obstacles detection on roads scenes using semantic segmentation for the safe navigation of autonomous vehicles. J Electron Imaging 31:061806

    Article  Google Scholar 

  69. Ashraf R, Afzal S, Rehman AU, Gul S, Baber J, Bakhtyar M et al (2020) Region-of-interest based transfer learning assisted framework for skin cancer detection. IEEE Access 8:147858–147871

    Article  Google Scholar 

  70. Qiu Q, Zhao J, Zheng Y (2022) Partial person re-identification using a pose-guided alignment network with mask learning. Appl Intell 52:1–16

    Article  Google Scholar 

  71. Sun C, Wang D, Lu H (2016) Person re-identification via distance metric learning with latent variables. IEEE Trans Image Process 26:23–34

    Article  MathSciNet  MATH  Google Scholar 

  72. Geng M, Wang Y, Xiang T, Tian Y (2016) Deep transfer learning for person re-identification. arXiv preprint arXiv:1611.05244

  73. Xie G, Wen X, Yuan L, Wang J, Guo C, Jia Y et al (2021) Pose-guided feature region-based fusion network for occluded person re-identification. Multimed Syst. https://doi.org/10.1007/s00530-021-00752-2

    Article  Google Scholar 

  74. Wei L, Zhang S, Gao W, Tian Q (2018) Person transfer gan to bridge domain gap for person re-identification. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 79–88

  75. Pei S, Fan X (2021) Global contrastive person re-identification. J Phys Conf Ser 1757:012035

    Article  Google Scholar 

  76. Li M, Zhu X, Gong S (2019) Unsupervised tracklet person re-identification. IEEE Trans Pattern Anal Mach Intell 42:1770–1782

    Article  Google Scholar 

  77. Hu Y, Yi D, Liao S, Lei Z, Li SZ (2004) Cross dataset person re-identification. In: Asian Conference on Computer Vision, pp 650–664

  78. Zhou S, Wang Y, Zhang F, Wu J (2021) Cross-view similarity exploration for unsupervised cross-domain person re-identification. Neural Comput Appl 33:4001–4011

    Article  Google Scholar 

  79. He L, Liang J, Li H, Sun Z (2018) Deep spatial feature reconstruction for partial person re-identification: alignment-free approach. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 7073–7082

  80. Kang JK, Lee MB, Yoon HS, Park KR (2021) AS-RIG: adaptive selection of reconstructed input by generator or interpolation for person re-identification in cross-modality visible and thermal images. IEEE Access 9:12055–12066

    Article  Google Scholar 

  81. Feng H, Chen M, Hu J, Shen D, Liu H, Cai D (2021) Complementary pseudo labels for unsupervised domain adaptation on person re-identification. IEEE Trans Image Process 30:2898–2907

    Article  Google Scholar 

  82. Bak S, Carr P. (2017) One-shot metric learning for person re-identification. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 2990–2999

  83. Figueira D, Bazzani L, Minh HQ, Cristani M, Bernardino A, Murino V (2013) Semi-supervised multi-feature learning for person re-identification. In: 2013 10th IEEE International Conference on Advanced Video and Signal Based Surveillance, pp 111–116

  84. Liu X, Song M, Tao D, Zhou X, Chen C, Bu J (2014) Semi-supervised coupled dictionary learning for person re-identification. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 3550–3557

  85. Zhu X, Jing X-Y, Yang L, You X, Chen D, Gao G et al (2017) Semi-supervised cross-view projection-based dictionary learning for video-based person re-identification. IEEE Trans Circuits Syst Video Technol 28:2599–2611

    Article  Google Scholar 

  86. Zhu F, Kong X, Fu H, Tian Q (2018) Pseudo-positive regularization for deep person re-identification. Multimed Syst 24:477–489

    Article  Google Scholar 

  87. Shao J, Ma X (2022) Hierarchical pseudo-label learning for one-shot person re-identification. Appl Intell 52:9225–9238

    Article  Google Scholar 

  88. Xin X, Wang J, Xie R, Zhou S, Huang W, Zheng N (2019) Semi-supervised person re-identification using multi-view clustering. Pattern Recognit 88:285–297

    Article  Google Scholar 

  89. Wu L, Wang Y, Yin H, Wang M, Shao L (2019) Few-shot deep adversarial learning for video-based person re-identification. IEEE Trans Image Process 29:1233–1245

    Article  MathSciNet  MATH  Google Scholar 

  90. Hanif M, Ling H, Tian W, Shi Y, Rauf M (2021) Re-ranking person re-identification using distance aggregation of k-nearest neighbors hierarchical tree. Multimed Tools Appl 80:8015–8038

    Article  Google Scholar 

  91. Zhu F, Kong X, Wu Q, Fu H, Li M (2018) A loss combination based deep model for person re-identification. Multimed Tools Appl 77:3049–3069

    Article  Google Scholar 

  92. Liu J, Zha Z-J, Chen X, Wang Z, Zhang Y (2019) Dense 3D-convolutional neural network for person re-identification in videos. ACM Trans Multimed Comput Commun Appl (TOMM) 5:1–19

    Google Scholar 

  93. Zhang Y, Ma B, Liu L, Yi X, Li M, Diao Y (2022) Self-paced uncertainty estimation for one-shot person re-identification. Appl Intell. https://doi.org/10.1007/s10489-022-04245-1

    Article  Google Scholar 

  94. Li M, Zhu X, Gong S (2018) Unsupervised person re-identification by deep learning tracklet association. In: Proceedings of the European Conference on Computer Vision (ECCV), pp 737–753

  95. Ma D, Zhou Y, Zhao J, Chen Y, Yao R, Chen H (2021) Video-based person re-identification by semi-supervised adaptive stepwise learning. Pattern Anal Appl 24:1769–1776

    Article  Google Scholar 

  96. Zheng Y, Zhou Y, Zhao J, Chen Y, Yao R, Liu B et al (2022) Clustering matters: sphere feature for fully unsupervised person re-identification. ACM Trans Multimed Comput Commun Appl (TOMM) 18:1–18

    Article  Google Scholar 

Download references

Funding

This work was supported by the National Research Foundation of Korea (NRF) grant funded by the Korea government (MSIT) (No. 2022R1F1A1063134) and also the Chung-Ang University Research Grants in 2022.

Author information

Authors and Affiliations

Authors

Contributions

E.B and S.G finalized the problem formulation; S.G., S.N, and M.M completed the formal analysis; E.B., M.B., and S.G carried out the implementation; E.B, M.B, and S.N finalized the experiments and results; M.B. and M.M finalized figures and typesetting; E.B., M.B, and M.M. wrote the paper; S.R. and S.Y. reviewed and finalized the draft.

Corresponding author

Correspondence to Seungmin Rho.

Ethics declarations

Conflict of interest

The authors declare there is no conflict of interest.

Ethical approval

Not applicable.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Batool, E., Gillani, S., Naz, S. et al. POSNet: a hybrid deep learning model for efficient person re-identification. J Supercomput 79, 13090–13118 (2023). https://doi.org/10.1007/s11227-023-05169-4

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11227-023-05169-4

Keywords

Navigation