Skip to main content
SpringerLink
Log in

A deep person re-identification model with multi visual-semantic information embedding

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

The local features of different body parts have been widely used to learn more discriminative representation for person re-identification, which act as either extra visual semantic information or auxiliary means to deal with the issue of misalignment and background bias. However, the existing person re-identification works mainly focuses on the common impact of multiple body parts while failing to explicitly explore the influence of body edge contour. As the edge contour is one of the most significant visual-semantic clues for object detection and person identification in the blurred scene, this paper intentionally explores the effect of edge contour clues on person re-identification and proposes a deep learning framework with multi visual-semantic information embedding, including body parts and edge contour. Meanwhile, we conceive a practical strategy which can effectively fuse the different body part features and reduce the dimensionality of features. Extensive experimental results on four benchmark data sets show that our model has achieved competitive accuracy compared to the state-of-the-art models.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

References

  1. Badrinarayanan V, Kendall A, Cipolla R (2017) SegNet: a deep convolutional encoder-decoder architecture for image segmentation. IEEE Trans Pattern Anal Mach Intell 39(12):2481–2495

    Article  Google Scholar 

  2. Chen L-C, Papandreou G, Kokkinos I, Murphy K, Yuille AL (2015) Semantic image segmentation with deep convolutional nets and fully connected CRFs. arXivpreprint arXiv: 1412.7062

  3. Chen L-C, Papandreous G, Schroff F, Adam H (2017) Rethinking Atrous Convolution for Semantic Image Segmentation. arXivpreprint arXiv: 1706:05587

  4. Chen W, Chen X, Zhang J, Huang K (2017) A multi-task deep network for person re-identification. AAAI 2017:3988–3994

    Google Scholar 

  5. Chen L-C, Papandreou G, Kokkinos I, Murphy K, Yuille AL (2018) DeepLab: semantic image segmentation with deep convolutional nets, Atrous convolution, and fully connected CRFs. IEEE Trans Pattern Anal Mach Intell 40(4):834–848

    Article  Google Scholar 

  6. Chen L-C, Zhu Y, Papandreou G, Schroff F, Adam H (2018) Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation. ECCV (7) 2018: 833–851

  7. Cheng DS, Cristani M, Stoppa M, Bazzani L, Murino V (2011) Custom pictorial structures for re-identification. BMVC 2011:1–11

    Google Scholar 

  8. Chu H, Qi M, Liu H, Jiang J (2019) Local region partition for person re-identification. Multimed Tools Appl 78:27067–27083. https://doi.org/10.1007/s11042-017-4817-4

    Article  Google Scholar 

  9. Das A, Chakraborty A, Roy-Chowdhury AK (2014) Consistent Re-identification in a Camera Network. ECCV (2) 2014: 330–345

  10. Fan X, Luo H, Zhang X, He L, Zhang C, Jiang W (2018) SCPNET: Spatial-Channel Parallelism Network for Joint Holistic and Partial Person Re-identification. ACCV(2) 2018: 19–34

  11. Fan X, Jiang W, Luo H, Fei M (2019) Deep hypersphere manifold embedding for person reidentification. J Vis Commun Image Represent 60:51–58. https://doi.org/10.1016/j.jvcir.2019.01.010

    Article  Google Scholar 

  12. Ganin Y, Lempitsky V (2014) N4-fields: neural network nearest neighbor fields for image transforms. ACCV 2014:536–551

    Google Scholar 

  13. Geng M, Wang Y, Xiang T, Tian Y (2016) Deep Transfer Learning for Person Re-identification.arXivpreprint arXiv:1611.05244

  14. Gong K, Liang X, Zhang D, Shen X, Liang L (2017) Look into person: self-supervised structure-sensitive learning and a new benchmark for human parsing. CVPR 2017:6757–6765

    Google Scholar 

  15. Gray D, Tao H (2008) Viewpoint Invariant Pedestrian Recognition with an Ensemble of Localized Features. ECCV (1) 2008: 262–275

  16. He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. CVPR 2016:770–778

    Google Scholar 

  17. Huang X, Ge Z, Jie Z, Yoshie O (2020) NMS by Pepresentative Region: Towards Crowded Pedestrain Detection by Proposal Pairing CVPR 2020. https://arxiv.org/pdf/2003.12729.pdf

  18. Hwang J-J, Liu T-L (2015) Pixel-wise deep learning for contour detection. arXiv preprint arXiv:1504.01989

  19. F. Iandola, M. Moskewicz, S. Karayev, R. Girshick, T. Darrell, and K. Keutzer (2014) DenseNet: Implementing Efficient ConvNet Descriptor Pyramids arXiv preprint arXiv: 1404.1869

  20. Kalayeh MM, Basaran E, Gokmen M, Kamasak ME, Shah M (2018) Human semantic parsing for person re-identification. CVPR 2018:1062–1071

    Google Scholar 

  21. Li W, Zhao R, Xiao T, Wang X (2014) DeepReID: deep filter pairing neural network for person re-identification. CVPR 2014:152–159

    Google Scholar 

  22. Li W, Zhu X, Gong S (2017) Person re-identification by deep joint learning of multi-loss classification. IJCAI 2017:2194–2200

    Google Scholar 

  23. Liang Z, Shen L, Lu T, Shengjin JW, Tian Q (2015) Scalable person re-identification: a benchmark. ICCV 2015:1116–1124

    Google Scholar 

  24. Liang Z, Huang Y, Lu H, Yang Y (2019) Pose-invariant embedding for deep person re-identification. IEEE Trans Image Processing 28(9):4500–4509

    Article  MathSciNet  Google Scholar 

  25. Lim J, Zitnick CL, Dollar P (2013) Sketch tokens: Alearned mid-level representation for contour and object detection. CVPR 2013:3158–3165

    Google Scholar 

  26. Lin G, Milan A, Shen C, Reid ID (2017) RefineNet: multi-path refinement networks for high-resolution semantic segmentation. CVPR 2017:5168–5177

    Google Scholar 

  27. Liu Y, Cheng M-M, Hu X, Bian J-W, Zhang L, Bai X, Tang J (2019) Richer convolutional features for edge detection. IEEE Trans Pattern Anal Mach Intell 41(8):1939–1946

    Article  Google Scholar 

  28. Long J, Shelhamer E, Darrell T (2017) Fully convolutional networks for semantic segmentation. IEEE Trans Pattern Anal Mach Intell 39(4):640–651

    Article  Google Scholar 

  29. Luo H, Jiang W, Zhang X, Fan X, Qian J, Zhang C (2019) AlignedReID++: dynamically matching local information for person re-identification. Pattern Recogn 94:53–61. https://doi.org/10.1016/j.patcog.2019.05.028

    Article  Google Scholar 

  30. Ma AJ, Yuen PC, Li J (2013) Domain transfer support vector ranking for person re-identification without target camera label information. ICCV 2013:3567–3574

    Google Scholar 

  31. Qi L, Huo J, Wang L, Shi Y, Gao Y (2018) MaskReID: A Mask Based Deep Ranking Neural Network for Person Re-Identification. arXiv preprint arXiv:1804.03864

  32. Saquib Sarfraz M, Schumann A, Eberle A, Stiefelhagen R (2018) A Pose-Sensitive Embedding for Person Re-Identification with Expanded Cross Neighborhood Re-Ranking CVPR 2018

  33. Sun L, Cheng Z, Yan Z, Liu P, Duckett T, Stolkin R (2018) A novel weakly-supervised approach for RGB-D-based nuclear waste object detection. IEEE Sensors J 19(9):3487–3500

    Article  Google Scholar 

  34. Sun Y, Zheng L, Yang Y, Tian Q, Wang S (2018) Beyond Part Models: Person Retrieval with Refined Part Pooling. ECCV (7) 2018:480–496

  35. Sun Y, Xu Q, Li Y, Zhang C, Li Y, Wang S, Sun J (2019) Perceive where to focus: learning visibility-aware part-level features for partial person re-identification. CVPR 2019:393–402

    Google Scholar 

  36. Wang T, Yang T, Danelljan M, Khan FS, Zhang X, Sun J (2020) Learning human-object interaction detection using interaction points. CVPR 2020 https://arxiv.org/pdf/2003.14023v1.pdf.

  37. Watson G, Bhalerao A (2020) Person re-identification combining deep features and attribute detection. Multimed Tools Appl 79:6463–6481. https://doi.org/10.1007/s11042-019-08499-9

    Article  Google Scholar 

  38. Wei L, Zhang S, Yao H, Gao W, Tian Q (2017) GLAD: Global-Local-Alignment Descriptor for Pedestrian Retrieval. ACM Multimedia 2017:420–428. https://doi.org/10.1145/3123266.3123279

    Article  Google Scholar 

  39. Wei L, Zhang S, Gao W, Tian Q (2018) Person transfer GAN to bridge domain gap for person re-identification. CVPR 2018:79–88

    Google Scholar 

  40. Xie S, Zhuowen T (2017) Holistically-nested edge detection. Int J Comput Vis 125(1–3):3–18. https://doi.org/10.1007/s11263-017-1004-z

    Article  MathSciNet  Google Scholar 

  41. Zhang X, Luo H, Fan X, Xiang W, Sun Y, Xiao Q, Jiang W, Sun J (2017) AlignedReID: Surpassing Human-Level Performance in Person Re-identification. arXiv preprint arXiv:1711.08184

  42. Zheng Z, Zheng L, Yang Y (2017) A discriminatively learned CNN embedding for person re-identification. Acm Transactions on Multimedia Computing Communications & Applications, 14(1)

  43. Zheng Z, Liang Z, Yang Y (2017) Unlabeled samples generated by GAN improve the person re-identification baseline in vitro. ICCV 2017:3774–3782

    Google Scholar 

  44. Zheng Z, Yang X, Yu Z, Liang Z, Yang Y, Kautz J (2019) Joint discriminative and generative learning for person re-identification. CVPR 2019:2138–2147

    Google Scholar 

  45. Zhou Q, Zhong B, Lan X, Sun G, Zhang Y, Gou M (2019) LRDNN: Local-refining based Deep Neural Network for Person Re-Identification with Attribute Discerning. IJCAI: 1041–1047

Download references

Acknowledgments

This work is financially supported in part by the National Science Foundation of China under Grant No.62072372, No.61972315, No.61973250, No.61902318, and Key Research and Development Program of Shaanxi (Program No.2019GY-012, No.2018SF-369). We are also grateful to the Shaanxi Science and Technology Innovation Team Support Project under grant agreement 2018TD-026.

Author information

Authors and Affiliations

  1. Department of Computer Science, Northwest University, Xian, China

    Xiaopei Wang, Xiaoxia Liu, Jun Guo, Jiaxiang Zheng, Pengfei Xu & Yun Xiao

  2. Shaanxi International Joint Research Centre for the Battery-free Internet of Tings, Xian, China

    Baoying Liu

Authors
  1. Xiaopei Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xiaoxia Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jun Guo
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jiaxiang Zheng
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Pengfei Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Yun Xiao
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Baoying Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jun Guo.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wang, X., Liu, X., Guo, J. et al. A deep person re-identification model with multi visual-semantic information embedding. Multimed Tools Appl 80, 6853–6870 (2021). https://doi.org/10.1007/s11042-020-09957-5

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-020-09957-5

Keywords

Search

Navigation