Abstract
Person re-identification refers to the process of recognizing a person across several non-overlapping cameras. It is becoming increasingly important in computer vision for real-world surveillance applications. However, the deployment of person-re-identification systems as a surveillance system raises various challenges in their performance. These challenges include limited labeled data, occlusions conditions, human body postures, as well as inter- and intra-class variations. Such challenges deteriorate the effectiveness of person-re-identification systems and lead to the extraction of less discriminative features. Hence, to address these problems, we proposed a hybrid deep learning model, namely POSNet (pseudo-labeled omni-scale network) for efficient person re-identification. The proposed method is referred to as a hybrid because it combines label estimate with modified omni-scale feature learning, i.e., spatiotemporal-assisted omni-scale feature extraction to accomplish person re-identification. To further enhance omni-scale feature learning, we have proposed soft-pool-assisted attention mechanisms during spatial learning. More precisely, soft-pool preserves more important features, and that features are further emphasized by spatial and channel attention layers. Following on, this omni-scale with soft-pool attention learning extracts the spatial information from all frames of videos, and later on, the temporal learning is incorporated using the LSTM model. To handle limited labeled data problems, the proposed hybrid model first assigns pseudo-labels to the unlabeled data and adopts a progressive learning strategy to retrain the model on both labeled and unlabeled data with improved feature extraction, i.e., modified omni-scale feature learning. Moreover, the proposed POSNet model is validated on two large video-based person re-identification datasets, namely MARS and DukeMTMC-Video. It is observed from the research findings that the proposed POSNet outperformed the existing studies with the highest mAP and rank@1 score of 83.7 and 90.3%, respectively.
Similar content being viewed by others
Data availability
The data are publicly available and can also be provided on request to the corresponding author.
References
Zheng L, Yang Y, Hauptmann AG (2016) Person re-identification: Past, present and future. arXiv preprint arXiv:1610.02984
Bialkowski A, Denman S, Sridharan S, Fookes C, Lucey P (2012) A database for person re-identification in multi-camera surveillance networks. In: 2012 International Conference on Digital Image Computing Techniques and Applications (DICTA), pp 1-8
Yadav A, Vishwakarma DK (2020) Person re-identification using deep learning networks: a systematic review. arXiv preprint arXiv:2012.13318
Ye M, Shen J, Lin G, Xiang T, Shao L, Hoi SC (2021) Deep learning for person re-identification: a survey and outlook. IEEE Trans Pattern Anal Mach Intell 44:2872–2893
Ahmed E, Jones M, Marks TK (2015) An improved deep learning architecture for person re-identification. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 3908–3916
Wu D, Zheng S-J, Zhang X-P, Yuan C-A, Cheng F, Zhao Y et al (2019) Deep learning-based methods for person re-identification: a comprehensive review. Neurocomputing 337:354–371
Li W, Zhao R, Xiao T, Wang X (2014) Deepreid: deep filter pairing neural network for person re-identification. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 152–159
Cheng D, Gong Y, Zhou S, Wang J, Zheng N (2016) Person re-identification by multi-channel parts-based cnn with improved triplet loss function. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 1335–1344
Tian Y, Li Q, Wang D, Wan B (2019) Robust joint learning network: improved deep representation learning for person re-identification. Multimed Tools Appl 78:24187–24203
Sun Y, Zheng L, Yang Y, Tian Q, Wang S (2018) Beyond part models: person retrieval with refined part pooling (and a strong convolutional baseline). In: Proceedings of the European Conference on Computer Vision (ECCV), pp 480–496
Xiao J, Li H, Qu G, Fujita H, Cao Y, Zhu J et al (2022) Hope: heatmap and offset for pose estimation. J Ambient Intell Humaniz Comput 13:2937–2949
Zhao H, Tian M, Sun S, Shao J, Yan J, Yi S et al (2017) Spindle net: person re-identification with human body region guided feature decomposition and fusion. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 1077–1085
Wei L, Zhang S, Yao H, Gao W, Tian Q (2017) Glad: global-local-alignment descriptor for pedestrian retrieval. In: Proceedings of the 25th ACM International Conference on Multimedia, pp 420–428
Zhu K, Guo H, Liu Z, Tang M, Wang J (2020) Identity-guided human semantic parsing for person re-identification. In: European Conference on Computer Vision, pp 346–363
Song C, Huang Y, Ouyang W, Wang L (2018) Mask-guided contrastive attention model for person re-identification. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 1179–1188
Zhao L, Li X, Zhuang Y, Wang J (2017) Deeply-learned part-aligned representations for person re-identification. In: Proceedings of the IEEE International Conference on Computer Vision, pp 3219–3228
Si J, Zhang H, Li C-G, Kuen J, Kong X, Kot AC et al (2018) Dual attention matching network for context-aware feature sequence based person re-identification. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 5363–5372
Liu J, Ni B, Yan Y, Zhou P, Cheng S, Hu J (2018) Pose transferrable person re-identification. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 4099–4108
Ma L, Sun Q, Georgoulis S, Van Gool L, Schiele B, Fritz M (2018) Disentangled person image generation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 99–108
Qian X, Fu Y, Xiang T, Wang W, Qiu J, Wu Y et al (2018) Pose-normalized image generation for person re-identification. In: Proceedings of the European Conference on Computer Vision (ECCV), pp 650–667
Ge Y, Li Z, Zhao H, Yin G, Yi S, Wang X (2018) Fd-gan: pose-guided feature distilling gan for robust person re-identification. In: Advances in Neural Information Processing Systems, vol 31
Gray D, Brennan S, Tao H (2007) Evaluating appearance models for recognition, reacquisition, and tracking. In: Proceeding IEEE International Workshop on Performance Evaluation for Tracking and Surveillance (PETS), pp 1–7
Ess A, Leibe B, Van Gool L (2007) Depth and appearance for mobile scene analysis. In: 2007 IEEE 11th International Conference on Computer Vision, pp 1–8
Schwartz WR, Davis LS (2009) Learning discriminative appearance-based models using partial least squares. In: 2009 XXII Brazilian Symposium on Computer Graphics and Image Processing, pp 322–329
Hirzer M, Beleznai C, Roth PM, Bischof H (2011) Person re-identification by descriptive and discriminative classification. In: Scandinavian Conference on Image analysis, pp 91–102
Cheng DS, Cristani M, Stoppa M, Bazzani L, Murino V (2011) Custom pictorial structures for re-identification. Bmvc 1:6
Prosser BJ, Zheng W-S, Gong S, Xiang T, Mary Q (2010) Person re-identification by support vector ranking. Bmvc 2:6
Wang T, Gong S, Zhu X, Wang S (2014) Person re-identification by video ranking. In: European Conference on Computer Vision, pp 688–703
Gevers T, Smeulders AW (1999) Color-based object recognition. Pattern Recognit 32:453–464
Zheng L, Shen L, Tian L, Wang S, Wang J, Tian Q (2015) Scalable person re-identification: a benchmark. In Proceedings of the IEEE International Conference on Computer Vision, pp 1116–1124
Dehghan A, Modiri Assari S, Shah M (2015) Gmmcp tracker: globally optimal generalized maximum multi clique problem for multiple object tracking. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 4091–4099
Felzenszwalb P, Girshick R, Mcallester D, Ramanan D (2013) DPM & Latent SVM. Course Febr
Ristani E, Solera F, Zou R, Cucchiara R, Tomasi C (2016) Performance measures and a data set for multi-target, multi-camera tracking. In: European Conference on Computer Vision, pp 17–35
Wu Y, Lin Y, Dong X, Yan Y, Ouyang W, Yang Y (2018) Exploit the unknown gradually: one-shot video-based person re-identification by stepwise learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 5177–5186
Gou M, Karanam S, Liu W, Camps O, Radke RJ (2017) DukeMTMC4ReID: a large-scale multi-camera person re-identification dataset. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp 10–19
Zheng Z, Zheng L, Yang Y (2017) Unlabeled samples generated by gan improve the person re-identification baseline in vitro. In: Proceedings of the IEEE International Conference on Computer Vision, pp 3754–3762
Gao S, Wang J, Lu H, Liu Z (2020) Pose-guided visible part matching for occluded person reid. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp 11744–11752
Zheng W-S, Gong S, Xiang T (2011) Person re-identification by probabilistic relative distance comparison. CVPR 2011:649–656
Gray D, Tao H (2008) Viewpoint invariant pedestrian recognition with an ensemble of localized features. In European Conference on Computer Vision, pp 262–275
Lin Y, Dong X, Zheng L, Yan Y, Yang Y (2019) A bottom-up clustering approach to unsupervised person re-identification. In Proceedings of the AAAI Conference on Artificial Intelligence, pp 8738–8745
Zang X, Li G, Gao W, Shu X (2022) Exploiting robust unsupervised video person re-identification. IET Image Process 16:729–741
Teng H, He T, Guo Y, Ding G (2020) A high-accuracy unsupervised person re-identification method using auxiliary information mined from datasets. arXiv preprint arXiv:2205.03124
Gratacos B (2006) A robust algorithm and associated QC for finding the anisotropy directions for converted wave data. In: 68th EAGE Conference and Exhibition Incorporating SPE EUROPEC 2006, pp cp-2–00333
Liu Z, Wang D, Lu H (2017) Stepwise metric promotion for unsupervised video person re-identification. In: Proceedings of the IEEE International Conference on Computer Vision, pp 2429–2438
Ye M, Ma AJ, Zheng L, Li J, Yuen PC (2017) Dynamic label graph matching for unsupervised video re-identification. In: Proceedings of the IEEE International Conference on Computer Vision, pp 5142–5150
Ma AJ, Li P (2015) Semi-supervised ranking for re-identification with few labeled image pairs. In: Asian Conference on Computer Vision, pp 598–613
Xiang S, Fu Y, Guan M, Liu T (2022) Learning from self-discrepancy via multiple co-teaching for cross-domain person re-identification. Mach Learn. https://doi.org/10.1007/s10994-022-06184-x
Ge Y, Chen D, Li H (2020) Mutual mean-teaching: Pseudo label refinery for unsupervised domain adaptation on person re-identification. arXiv preprint arXiv:2001.01526
Xiang S, Fu Y, Liu T (2020) Progressive learning with style transfer for distant domain adaptation. IET Image Process 14:3527–3535
Cho Y, Kim WJ, Hong S, Yoon S-E (2022) Part-based pseudo label refinement for unsupervised person re-identification. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp 7308–7318
Nambiar A, Bernardino A, Nascimento JC (2019) Gait-based person re-identification: a survey. ACM Comput Surv (CSUR) 52:1–34
Miao J, Wu Y, Yang Y (2021) Identifying visible parts via pose estimation for occluded person re-identification. IEEE Trans Neural Netw Learn Syst. https://doi.org/10.1109/TNNLS.2021.3059515
Wang Y, Liao S, Shao L (2020) Surpassing real-world source training data: random 3D characters for generalizable person re-identification. In Proceedings of the 28th ACM International Conference on Multimedia, pp 3422–3430
Almasawa MO, Elrefaei LA, Moria K (2019) A survey on deep learning-based person re-identification systems. IEEE Access 7:175228–175247
Lin Y, Zheng L, Zheng Z, Wu Y, Hu Z, Yan C et al (2019) Improving person re-identification by attribute and identity learning. Pattern Recognit 95:151–161
Schumann A, Stiefelhagen R (2017) Person re-identification by deep learning attribute-complementary information. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp 20–28
Ma T, Yang M, Rong H, Qian Y, Tian Y, Al-Nabhan N (2021) Dual-path CNN with max gated block for text-based person re-identification. Image Vis Comput 111:104168
Wang G, Fang Y, Wang J, Sun J (2016) Extensive comparison of visual features for person re-identification. In: Proceedings of the International Conference on Internet Multimedia Computing and Service, pp 192–196
Tome P, Fierrez J, Vera-Rodriguez R, Nixon MS (2014) Soft biometrics and their application in person recognition at a distance. IEEE Trans Inf Forensics Secur 9:464–475
Fan H, Zheng L, Yan C, Yang Y (2018) Unsupervised person re-identification: clustering and fine-tuning. ACM Trans Multimed Comput Commun Appl (TOMM) 14:1–18
Stergiou A, Poppe R, Kalliatakis G (2019) Refining activation downsampling with SoftPool. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp 10357–10366
Zhou K, Yang Y, Cavallaro A, Xiang T (2019) Omni-scale feature learning for person re-identification. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp 3702–3712
Bak S, Corvee E, Bremond F, Thonnat M (2010) Person re-identification using spatial covariance regions of human body parts. In: 2010 7th IEEE International Conference on Advanced Video and Signal Based Surveillance, pp 435–440
Imani Z, Soltanizadeh H (2019) Local binary pattern, local derivative pattern and skeleton features for RGB-D person re-identification. Natl Acad Sci Lett 42:233–238
Wang S, Xu X, Liu L, Tian J (2020) Multi-level feature fusion model-based real-time person re-identification for forensics. J Real-Time Image Proc 17:73–81
Fayyaz M, Yasmin M, Sharif M, Shah JH, Raza M, Iqbal T (2020) Person re-identification with features-based clustering and deep features. Neural Comput Appl 32:10519–10540
Bukhari M, Bajwa KB, Gillani S, Maqsood M, Durrani MY, Mehmood I et al (2020) An efficient gait recognition method for known and unknown covariate conditions. IEEE Access 9:6465–6477
Yasmin S, Durrani MY, Gillani S, Bukhari M, Maqsood M, Zghaibeh M (2022) Small obstacles detection on roads scenes using semantic segmentation for the safe navigation of autonomous vehicles. J Electron Imaging 31:061806
Ashraf R, Afzal S, Rehman AU, Gul S, Baber J, Bakhtyar M et al (2020) Region-of-interest based transfer learning assisted framework for skin cancer detection. IEEE Access 8:147858–147871
Qiu Q, Zhao J, Zheng Y (2022) Partial person re-identification using a pose-guided alignment network with mask learning. Appl Intell 52:1–16
Sun C, Wang D, Lu H (2016) Person re-identification via distance metric learning with latent variables. IEEE Trans Image Process 26:23–34
Geng M, Wang Y, Xiang T, Tian Y (2016) Deep transfer learning for person re-identification. arXiv preprint arXiv:1611.05244
Xie G, Wen X, Yuan L, Wang J, Guo C, Jia Y et al (2021) Pose-guided feature region-based fusion network for occluded person re-identification. Multimed Syst. https://doi.org/10.1007/s00530-021-00752-2
Wei L, Zhang S, Gao W, Tian Q (2018) Person transfer gan to bridge domain gap for person re-identification. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 79–88
Pei S, Fan X (2021) Global contrastive person re-identification. J Phys Conf Ser 1757:012035
Li M, Zhu X, Gong S (2019) Unsupervised tracklet person re-identification. IEEE Trans Pattern Anal Mach Intell 42:1770–1782
Hu Y, Yi D, Liao S, Lei Z, Li SZ (2004) Cross dataset person re-identification. In: Asian Conference on Computer Vision, pp 650–664
Zhou S, Wang Y, Zhang F, Wu J (2021) Cross-view similarity exploration for unsupervised cross-domain person re-identification. Neural Comput Appl 33:4001–4011
He L, Liang J, Li H, Sun Z (2018) Deep spatial feature reconstruction for partial person re-identification: alignment-free approach. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 7073–7082
Kang JK, Lee MB, Yoon HS, Park KR (2021) AS-RIG: adaptive selection of reconstructed input by generator or interpolation for person re-identification in cross-modality visible and thermal images. IEEE Access 9:12055–12066
Feng H, Chen M, Hu J, Shen D, Liu H, Cai D (2021) Complementary pseudo labels for unsupervised domain adaptation on person re-identification. IEEE Trans Image Process 30:2898–2907
Bak S, Carr P. (2017) One-shot metric learning for person re-identification. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 2990–2999
Figueira D, Bazzani L, Minh HQ, Cristani M, Bernardino A, Murino V (2013) Semi-supervised multi-feature learning for person re-identification. In: 2013 10th IEEE International Conference on Advanced Video and Signal Based Surveillance, pp 111–116
Liu X, Song M, Tao D, Zhou X, Chen C, Bu J (2014) Semi-supervised coupled dictionary learning for person re-identification. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 3550–3557
Zhu X, Jing X-Y, Yang L, You X, Chen D, Gao G et al (2017) Semi-supervised cross-view projection-based dictionary learning for video-based person re-identification. IEEE Trans Circuits Syst Video Technol 28:2599–2611
Zhu F, Kong X, Fu H, Tian Q (2018) Pseudo-positive regularization for deep person re-identification. Multimed Syst 24:477–489
Shao J, Ma X (2022) Hierarchical pseudo-label learning for one-shot person re-identification. Appl Intell 52:9225–9238
Xin X, Wang J, Xie R, Zhou S, Huang W, Zheng N (2019) Semi-supervised person re-identification using multi-view clustering. Pattern Recognit 88:285–297
Wu L, Wang Y, Yin H, Wang M, Shao L (2019) Few-shot deep adversarial learning for video-based person re-identification. IEEE Trans Image Process 29:1233–1245
Hanif M, Ling H, Tian W, Shi Y, Rauf M (2021) Re-ranking person re-identification using distance aggregation of k-nearest neighbors hierarchical tree. Multimed Tools Appl 80:8015–8038
Zhu F, Kong X, Wu Q, Fu H, Li M (2018) A loss combination based deep model for person re-identification. Multimed Tools Appl 77:3049–3069
Liu J, Zha Z-J, Chen X, Wang Z, Zhang Y (2019) Dense 3D-convolutional neural network for person re-identification in videos. ACM Trans Multimed Comput Commun Appl (TOMM) 5:1–19
Zhang Y, Ma B, Liu L, Yi X, Li M, Diao Y (2022) Self-paced uncertainty estimation for one-shot person re-identification. Appl Intell. https://doi.org/10.1007/s10489-022-04245-1
Li M, Zhu X, Gong S (2018) Unsupervised person re-identification by deep learning tracklet association. In: Proceedings of the European Conference on Computer Vision (ECCV), pp 737–753
Ma D, Zhou Y, Zhao J, Chen Y, Yao R, Chen H (2021) Video-based person re-identification by semi-supervised adaptive stepwise learning. Pattern Anal Appl 24:1769–1776
Zheng Y, Zhou Y, Zhao J, Chen Y, Yao R, Liu B et al (2022) Clustering matters: sphere feature for fully unsupervised person re-identification. ACM Trans Multimed Comput Commun Appl (TOMM) 18:1–18
Funding
This work was supported by the National Research Foundation of Korea (NRF) grant funded by the Korea government (MSIT) (No. 2022R1F1A1063134) and also the Chung-Ang University Research Grants in 2022.
Author information
Authors and Affiliations
Contributions
E.B and S.G finalized the problem formulation; S.G., S.N, and M.M completed the formal analysis; E.B., M.B., and S.G carried out the implementation; E.B, M.B, and S.N finalized the experiments and results; M.B. and M.M finalized figures and typesetting; E.B., M.B, and M.M. wrote the paper; S.R. and S.Y. reviewed and finalized the draft.
Corresponding author
Ethics declarations
Conflict of interest
The authors declare there is no conflict of interest.
Ethical approval
Not applicable.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Batool, E., Gillani, S., Naz, S. et al. POSNet: a hybrid deep learning model for efficient person re-identification. J Supercomput 79, 13090–13118 (2023). https://doi.org/10.1007/s11227-023-05169-4
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11227-023-05169-4