Skip to main content
SpringerLink
Log in

Combination of validity aggregation and multi-scale feature for person re-identification

  • Original Research
  • Published:
Journal of Ambient Intelligence and Humanized Computing Aims and scope Submit manuscript

Abstract

Person re-identification (re-ID) gradually has attracted attention of the industry and academe. It has a great application future in computer vision with the development of deep learning. The main challenges in the process of studying re-ID come from different camera angles, occlusion, and person’s posture changes. How to extract a powerful person descriptor is a fundamental problem in re-ID task, which is still an open topic today. In this study, we propose a Validity aggregation and multi-scale feature extraction network (VMSFEN), based on global and local features to convey more interesting information. In order to tackle local feature misalignment and feature pairs contribution, a novel strategy that combines cross-alignment and validity aggregation strategy is embedded into our model. Cross-alignment aims to obtain the same semantic features according to max semantic features. Validity aggregation provides appropriate weight to each matched local feature pair. Finally, we integrate the learned feature pairs with calculated weight, and introduce it into the triplet loss function. The approach of this work achieves 96.3% Rank-1 and 88.6% mAP on Market1501, 89.9% Rank-1 and 80.3% mAP on the DukeMTMC-reID, 80.1% Rank-1 and 76.3% mAP on the CUHK03. These findings prove that VMSFEN is an efficient network in re-ID study.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

References

  • Borgia A, Hua Y, Kodirov E, Robertson NM (2018) Cross-view discriminative feature learning for person re-identification. IEEE Trans Image Process 27(11):5338–5349

    Article  MathSciNet  Google Scholar 

  • Chang X, Hospedales TM, Xiang T (2018) Multi-level factorisation net for person re-identification. In: 2018 IEEE/CVF Conference on computer vision and pattern recognition, pp 2109–2118. https://doi.org/10.1109/CVPR.2018.00225

  • Chang H, Zhao D, Wu CH et al (2020) Visualization of spatial matching features during deep person re-identification. J Ambient Intell Humaniz Comput

  • Chen Y, Zhu X, Gong S (2017) Person re-identification by deep learning multi-scale representations. Proc IEEE Int Conf Comput vis Workshops 2017:2590–2600

    Google Scholar 

  • Chen YC, Zhu X, Zheng WS et al (2018) Person re-identification by camera correlation aware feature augmentation. IEEE Trans Pattern Anal Mach Intell 40(2):392–408

    Article  Google Scholar 

  • Chen D, Chen P, Yu X, Cao M, Jia T (2019) Deeply-learned spatial alignment for person re-identification. IEEE Access. https://doi.org/10.1109/ACCESS.2019.2945353

    Article  Google Scholar 

  • Cheng D, Gong Y, Zhou S et al (2016) Person re-identification by multi-channel parts-based CNN with improved triplet loss function. Computer vision and pattern recognition. IEEE

  • Ding Y, Fan H, Xu M et al (2020) Adaptive exploration for unsupervised person re-identification. ACM Trans Multimedia Comput Commun Appl. 16(1):19. https://doi.org/10.1145/3369393

    Article  Google Scholar 

  • Gao F, Jin Y, Ge Y et al (2020) Occluded person re-identification based on feature fusion and sparse reconstruction. Multimed Tools Appl 6:1–18

    Google Scholar 

  • Harel J, Koch C, Perona P (2007) Graph-based visual saliency. Adv Neural Inf Process Syst 2007:545–552

    Google Scholar 

  • Hermans A, Beyer L, Leibe B (2017) In defense of the triplet loss for person re-identification. https://arxiv.org/abs/1703.07737v2

  • Huang H, Yang W, Chen X, Zhao X, Huang K, Lin J, Huang G, Du D (2018) EANet: enhancing alignment for cross-domain person re-identification. ArXiv, abs/1812.11369

  • Huang Z, Yu Z, Li Y et al (2019) Contribution-based multi-stream feature distance fusion method with k-distribution re-ranking for person re-identification. IEEE Access 7:35631–35644. https://doi.org/10.1109/ACCESS.2019.2904278

    Article  Google Scholar 

  • Jose C, Fleuret F (2016) Scalable metric learning via weighted approximate rank component analysis. Eur Conf Comput vis 2016:875–890

    Google Scholar 

  • Kalayeh MM, Basaran E, Gökmen M et al (2018) Human semantic parsing for person re-identification. Proc IEEE Conf Comput vis Pattern Recognit 2018:1062–1071

    Google Scholar 

  • Karanam S, Gou M, Wu Z et al (2016) A comprehensive evaluation and benchmark for person re-identification: Features Metrics Datasets 2(3): 5. arXiv preprint arXiv: 1605.09653

  • Koestinger M, Hirzer M, Wohlhart P et al (2012) Large scale metric learning from equivalence constraints. IEEE Conf Comput vis Pattern Recognit 2012:2288–2295

    Google Scholar 

  • Li W, Zhao R, Xiao T et al (2014) DeepReID: Deep filter pairing neural network for person re-identification. Computer vision and pattern recognition. IEEE

  • Li W, Zhu X, Gong S (2017a) Person re-identification by deep joint learning of multi-loss classification. arXiv preprint arXiv: 1705.04724

  • Li D, Chen X, Zhang Z, et al (2017b) Learning Deep context-aware features over body and latent parts for person re-identification. In: 2017 IEEE Conference on computer vision and pattern recognition (CVPR), pp 7398–7407. https://doi.org/10.1109/CVPR.2017.782

  • Li Z, Jin Y, Li Y et al (2019) Learning part-alignment feature for person re-identification with spatial-temporal-based re-ranking method. World Wide Web 23:1907–1923. https://doi.org/10.1007/s11280-019-00734-5

    Article  Google Scholar 

  • Li R, Zhang B, Teng Z et al (2020) A divide-and-unite deep network for person re-identification. Appl Intell 2020:1–13

    Google Scholar 

  • Liang Z, Yujia H, Huchuan L et al (2019) Pose-invariant embedding for deep person re-identification. IEEE Trans Image Process 28(9):4500–4509. https://doi.org/10.1109/TIP.2019.2910414

    Article  MathSciNet  MATH  Google Scholar 

  • Liao S, Hu Y, Zhu Xiangyu and Li S (2015) Person re-identification by Local Maximal Occurrence representation and metric learning, IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2015, pp. 2197–2206. https://doi.org/10.1109/CVPR.2015.7298832

  • Liu X, Zhang S, Huang Q, Gao W (2018) RAM: a region-aware deep model for vehicle re-identification. 2018 IEEE international conference on multimedia and expo (ICME), San Diego, CA, pp 1–6. https://doi.org/10.1109/ICME.2018.8486589

  • Liu M, Yan X, Wang C et al (2020) Segmentation mask-guided person image generation. Appl Intell 2020:1–16

    Article  Google Scholar 

  • Luo H, Jiang W, Zhang X et al (2019) AlignedReID++: dynamically matching local information for person re-identification. Pattern Recognit 94:53–61

    Article  Google Scholar 

  • Noh H, Araujo A, Sim J et al (2017) Large-scale image retrieval with attentive deep local features. Proc IEEE Int Conf Comput vis 2017:3456–3465

    Google Scholar 

  • Poap D, Srivastava G (2020) Neural image reconstruction using a heuristic validation mechanism. Neural Comput Appl 33:10787–10797. https://doi.org/10.1007/s00521-020-05046-8

    Article  Google Scholar 

  • Połap D, Włodarczyk-Sielicka M, Wawrzyniak N (2021) Automatic ship classification for a riverside monitoring system using a cascade of artificial intelligence techniques including penalties and rewards. ISA Trans. https://doi.org/10.1016/j.isatra.2021.04.003

    Article  Google Scholar 

  • Schroff F, Kalenichenko D, Philbin J (2015) FaceNet: a unified embedding for face recognition and clustering, 2015 IEEE conference on computer vision and pattern recognition (CVPR), Boston, MA, pp 815–823. https://doi.org/10.1109/CVPR.2015.7298682

  • Song HO, Xiang Y, Jegelka S et al (2016) Deep metric learning via lifted structured feature embedding. In: 2016 IEEE Conference on computer vision and pattern recognition (CVPR), pp 4004–4012. https://doi.org/10.1109/CVPR.2016.434

  • Srivastava N, Hinton G, Krizhevsky A et al (2014) Dropout: a simple way to prevent neural networks from overfitting. J Mach Learn Res 15(1):1929–1958

    MathSciNet  MATH  Google Scholar 

  • Su C, Li J, Zhang S et al (2017) Pose-driven deep convolutional model for person re-identification. In: 2017 IEEE international conference on computer vision. IEEE

  • Sun Y, Zheng L, Yang Y et al (2018) Beyond part models: person retrieval with refined part pooling (and a strong convolutional baseline). In: Ferrari V, Hebert M, Sminchisescu C, Weiss Y (eds) Computer Vision – ECCV 2018. ECCV 2018. Lecture notes in computer science, vol 11208. Springer, Cham. https://doi.org/10.1007/978-3-030-01225-0_30

    Chapter  Google Scholar 

  • Tahboub K, Delgado B, Delp EJ (2016) Person re-identification using a patch-based appearance model. IEEE international conference on image processing (ICIP). IEEE

  • Varior RR, Haloi M, Wang G (2016a) Gated siamese convolutional neural network architecture for human re-identification. In: Leibe B, Matas J, Sebe N, Welling M (eds) Computer vision – ECCV 2016. ECCV 2016. Lecture Notes in computer science, vol 9912. Springer, Cham. https://doi.org/10.1007/978-3-319-46484-8_48

    Chapter  Google Scholar 

  • Varior RR, Shuai B, Lu J et al (2016b) A siamese long short-term memory architecture for human re-identification

  • Wang Y, Wang Z, Jia W et al (2018a) Joint learning of body and part representation for person re-identification. IEEE Access 6:44199-44210. https://doi.org/10.1109/ACCESS.2018.2864588

    Article  Google Scholar 

  • Wang G, Yuan Y, Chen X et al (2018b) Learning discriminative features with multiple granularities for person re-identification. In: Proceedings of the 26th ACM international conference on Multimedia (MM'18). Association for Computing Machinery, New York, NY, USA, pp 274–282. https://doi.org/10.1145/3240508.3240552

  • Wang C, Zhang Q, Huang C et al (2018c) Mancs: a multi-task attentional network with curriculum sampling for person re-identification. In: Ferrari V, Hebert M, Sminchisescu C, Weiss Y (eds) Computer Vision – ECCV 2018. ECCV 2018. Lecture Notes in Computer Science, vol 11208. Springer, Cham. https://doi.org/10.1007/978-3-030-01225-0_23

    Chapter  Google Scholar 

  • Wei S-E, Ramakrishna V, Kanade T et al (2016) Convolutional pose machines. Proc IEEE Conf Comput vis Pattern Recognit 2016:4724–4732

    Google Scholar 

  • Wei L, Zhang S, Yao H et al (2019) GLAD: global-local-alignment descriptor for scalable person re-identification. IEEE Trans Multimed 21(4):986–999

    Article  Google Scholar 

  • Xie H, Jiang W, Luo H et al (2020) Model compression via pruning and knowledge distillation for person re-identification. J Ambient Intell Human Comput 12:2149–2161. https://doi.org/10.1007/s12652-020-02312-4

    Article  Google Scholar 

  • Yao H, Zhang S, Hong R et al (2019) Deep representation learning with part loss for person re-identification. IEEE Trans Image Process 28(6):2860–2871

    Article  MathSciNet  MATH  Google Scholar 

  • Ye M, Yuen PC (2020) PurifyNet: a robust person re-identification model with noisy labels. IEEE Trans Inf Forensics Secur 15(99):2655–2666

    Article  Google Scholar 

  • Yi D, Lei Z, Liao S et al (2014) Deep metric learning for person re-identification. International conference on pattern recognition. IEEE Computer Society

  • Zeng M, Tian C, Wu Z (2018) Person Re-identification with Hierarchical Deep Learning Feature and efficient XQDA Metric. In: Proceedings of the 26th ACM international conference on Multimedia (MM'18). Association for Computing Machinery, New York, NY, USA, pp 1838–1846. https://doi.org/10.1145/3240508.3240717

  • Zhang Z, Lan C, Zeng W, Chen Z (2018a) Densely semantically aligned person re-identification. https://arxiv.org/abs/1812.08967v1

  • Zhang Y, Xiang T, Hospedales TM et al (2018b) Deep mutual learning. Proc IEEE Conf Comput vis Pattern Recognit 2018:4320–4328

    Google Scholar 

  • Zhang Z, Xie Y, Li D et al (2020) Learning to align via wasserstein for person re-identification. IEEE Trans Image Process 29:7104–7116. https://doi.org/10.1109/TIP.2020.2998931

    Article  MATH  Google Scholar 

  • Zhao L, Li X, Zhuang Y et al (2017a) Deeply-learned part-aligned representations for person re-identification. Proc IEEE Int Conf Comput vis 2017:3219–3228

    Google Scholar 

  • Zhao H, Tian M, Sun S et al (2017b) Spindle net: person re-identification with human body region guided feature decomposition and fusion. In: 2017 IEEE conference on computer vision and pattern recognition (CVPR). IEEE

  • Zheng W-S, Gong S, Xiang T (2012) Reidentification by relative distance comparison. IEEE Trans Pattern Anal Mach Intell 35(3):653–668

    Article  Google Scholar 

  • Zheng Z ,  Zheng L ,  Yang Y (2018) A discriminatively learned CNN embedding for person re-identification. ACM Trans Multimed Comput Commun Appl 14(1). arXiv:1611.05666

  • Zhong Z, Zheng L, Cao D and Li S (2017) Re-ranking person re-identification with k-reciprocal encoding, IEEE conference on computer vision and pattern recognition (CVPR) Honolulu, HI, pp 3652–3661. https://doi.org/10.1109/CVPR.2017.389

  • Zhong Z, Tongzhen S, Shuang L (2018) Integration Convolutional Neural Network for Person Re-Identification in Camera Networks, in IEEE Access, vol. 6, pp. 36887–36896, 2018, doi: 10.1109/ACCESS.2018.2852712.

    Article  Google Scholar 

  • Zhou S, Wang J, Meng D, Liang Y, Gong Y, Zheng N (2019) Discriminative feature learning with foreground attention for person re-identification. IEEE Trans Image Process. https://doi.org/10.1109/TIP.2019.2908065

    Article  MathSciNet  MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. School of Microelectronics and Communication Engineering, Chongqing University, Chongqing, 400044, People’s Republic of China

    Zhi-yong Huang, Wen-cheng Qin, Fen Luo, Tian-hui Guan, Fang Xie, Shu Han & Da-ming Sun

Authors
  1. Zhi-yong Huang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Wen-cheng Qin
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Fen Luo
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Tian-hui Guan
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Fang Xie
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Shu Han
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Da-ming Sun
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Zhi-yong Huang.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Annex

Annex

See Tables 4, 5 and 6.

Table 4 Some abbreviations in the paper
Full size table
Table 5 Parameters setting in the paper
Full size table
Table 6 Comparison of three re-ID datasets
Full size table

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Huang, Zy., Qin, Wc., Luo, F. et al. Combination of validity aggregation and multi-scale feature for person re-identification. J Ambient Intell Human Comput 14, 3353–3368 (2023). https://doi.org/10.1007/s12652-021-03473-6

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12652-021-03473-6

Keywords

Search

Navigation