Skip to main content
SpringerLink
Log in

Deep feature embedding learning for person re-identification based on lifted structured loss

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Person re-identification (re-id) aims at matching the same individual in videos captured by multiple cameras, and much progress has been made in recent years due to large scale pedestrian data sets and deep learning-based techniques. In this paper, we propose deep feature embedding learning for person re-id based on lifted structured loss. Triplet loss is commonly used in deep neural networks for person re-id. However, the triplet loss-based framework is not able to make full use of the batch information, and thus needs to choose hard negative samples manually that is time-consuming. To address this problem, we adopt lifted structured loss for deep neural networks that makes the network learn better feature embedding by minimizing intra-class variation and maximizing inter-class variation. Extensive experiments on Market-1501, CUHK03, CUHK01 and VIPeR data sets demonstrate the superior performance of the proposed method over state-of-the-arts in terms of the cumulative match curve (CMC) metric.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

References

  1. Barbosa IG, Cristani M, Caputo B, Rognhaugena A, Theoharis T (2017) Looking beyond appearances: Synthetic training data for deep cnns in re-identification. Comput Vis Image Underst 167:50–62

    Article  Google Scholar 

  2. Chen S-Z, Guo C-C, Lai J-H (2016) Deep ranking for person re-identification via joint representation learning. IEEE Trans Image Process 25(5):2353–2367

    Article  MathSciNet  Google Scholar 

  3. Chen D, Yuan Z, Chen B, Zheng N (2016) Similarity learning with spatial constraints for person re-identification. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1268–1277

  4. Chen W, Chen X, Zhang J, Huang K (2017) Beyond triplet loss: A deep quadruplet network for person re-identification. In: Proc. IEEE conference on computer vision and pattern recognition (CVPR)

  5. Cheng D, Gong Y, Zhou S, Wang J, Zheng N (2016) Person re-identification by multi-channel parts-based cnn with improved triplet loss function. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR), pp 1335–1344

  6. Ding S, Lin L, Wang G, Chao H (2015) Deep feature learning with relative distance comparison for person re-identification. Pattern Recogn 48(10):2993–3003

    Article  Google Scholar 

  7. Felzenszwalb PF, Girshick R, McAllester D, Ramanan D (2010) Object detection with discriminatively trained partbased models. IEEE Trans Pattern Anal Mach Intell 32(9):1627–1645

    Article  Google Scholar 

  8. Geng M, Wang Y, Xiang T, Tian Y (2016) Deep transfer learning for person re-identification. arXiv:1611.05244zz

  9. Girshick R, Donahue J, Darrell T, Malik J (2014) Rich feature hierarchies for accurate object detection and semantic segmentation. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR), pp 580–587

  10. Gray D, Brennan S, Tao H (2007) Evaluating appearance models for recognition, reacquisition, and tracking. In: Proc. IEEE international workshop on performance evaluation for tracking and surveillance (PETS), vol 3, pp 1–7

  11. Guo C-C, Chen S-Z, Lai J-H, Hu X-J, Shi S-C (2014) Multi-shot person re-identification with automatic ambiguity inference and removal. In: Proc. 22nd international conference on pattern recognition (ICPR), pp 3540–3545

  12. He K, Zhang X, Ren S, Sun J (2015) Deep residual learning for image recognition. arXiv:1512.03385

  13. He Z, Zhang Z, Jung C (2018) Deep feature embedding learning for person re-identification using lifted structured loss. In: Proceedings of IEEE conference on acoustics, speech and signal processing (ICASSP), pp 1957–1961

  14. Hermans A, Beyer L, Leibe B (2017) In defense of the triplet loss for person re-identification. arXiv:1703.07737

  15. Jin H, Wang X, Liao S, Li SZ (2017) Deep person re-identification with improved embedding. arXiv:1705.03332

  16. Jones M, Marks T (2015) An improved deep learning architecture for person re-identification. In: Proc. IEEE conference on computer vision and pattern recognition (CVPR)

  17. Karen S, Zisserman A (2014) Very deep convolutional networks for large-scale image recognition. arXiv:1409.1556

  18. Khamis S, Kuo C-H, Singh VK, Shet V D, Davis LS (2014) Joint learning for attribute-consistent person re-identification. In: Proc. ECCV Workshops, vol 3, pp 134–146

  19. Koestinger M, Hirzer M, Wohlhart P, Roth PM, Bischof H (2012) Large scale metric learning from equivalence constraints. In: Proc. IEEE conference on computer vision and pattern recognition (CVPR), pp 2288–2295

  20. Krizhevsky A, Sutskever I, Hinton GE (2012) Imagenet classification with deep convolutional neural networks. In: Advances in neural information processing systems (NIPS), pp 1097–1105

  21. Li W, Wang X (2013) Locally aligned feature transforms across views. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR), pp 3594-3601

  22. Li W, Zhao R, Xiao T, Wang X (2014) Deepreid, Deep filter pairing neural network for person re-identification. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR)

  23. Liao S, Hu Y, Zhu X, Li SZ (2015) Person re-identification by local maximal occurrence representation and metric learning. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR), pp 2197–2206

  24. Liu H, Feng J, Qi M, Jiang J, Yan S (2017) End-to-end comparative attention networks for person re-identification. IEEE Trans Image Process 26(7):3492–3506

    Article  MathSciNet  Google Scholar 

  25. Liu X, Song M, Zhao Q, Tao D, Chen C, Bu J (2012) Attribute-restricted latent topic model for person reidentification. Pattern Recogn 45(12):4204–4213

    Article  Google Scholar 

  26. Nanda A, Chauhan DS, Sa PK, Bakshi S (2017) Illumination and scale invariant relevant visual features with hypergraph-based learning for multi-shot person re-identification. Multimedia Tools and Applications, pp 1–26

  27. Nanda A, Sa PK, Choudhury SK, Bakshi S, Majhi B (2017) A Neuromorphic Person Re-Identification Framework for Video Surveillance. IEEE Access 5:6471–6482

    Google Scholar 

  28. Paisitkriangkrai S, Shen C, van den Hengel A (2015) Learning to rank in person re-identification with metric ensembles. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR), pp 1846–1855

  29. Schroff F, Kalenichenko D, Philbin J (2015) Facenet: A unified embedding for face recognition and clustering. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR), pp 815–823

  30. Shi H, Yang Y, Zhu X, Liao S, Lei Z, Zheng W, Li S Z (2016) Embedding deep metric for person re-identification: A study against large variations. In: Proc. European conference on computer vision (ECCV), pp 732–748, Springer

  31. Song H, Xiang Y, Jegelka S, Savarese S (2016) Deep metric learning via lifted structured feature embedding. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 4004–4012

  32. Subramaniam A, Chatterjee M, Mittal A (2016) Deep neural networks with inexact matching for person re-identification. In: Advances in neural information processing systems (NIPS), pp 2667–2675

  33. Varior RR, Haloi M, Wang G (2016) Gated siamese convolutional neural network architecture for human re-identification. In: Proc. European conference on computer vision (ECCV), pp 791–808, Springer

  34. Varior RR, Shuai B, Lu J, Xu D, Wang G (2016) A siamese long short-term memory architecture for human re-identification. In: Proc. European Conference on Computer Vision, pp 135–153, Springer

  35. Wang F, Zuo W, Lin L, Zhang D, Zhang L (2016) Joint learning of single-image and cross-image representations for person re-identification. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR), pp 1288–1296

  36. Weinberger KQ, Saul LK (2009) Distance metric learning for large margin nearest neighbor classification. J Mach Learn Res 10:207–244

    MATH  Google Scholar 

  37. Xiao T, Li H, Ouyang W, Wang X (2016) Learning deep feature representations with domain guided dropout for person re-identification. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR), pp 1249–1258

  38. Xiong F, Gou M, Camps O, Sznaier M (2014) Person re-identification using kernel-based metric learning methods. In: Proc. European conference on computer vision (ECCV), pp 1–16, Springer

  39. Yi D, Lei Z, Liao S, Li SZ (2014) Deep metric learning for person re-identification. In: Proc. 22nd international conference on pattern recognition (ICPR), pp 34–39

  40. Zhang L, Xiang T, Gong S (2016) Learning a discriminative null space for person re-identification. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1239–1248

  41. Zhang Y, Li B, Lu H, Irie A, Ruan X (2016) Sample specific svm learning for person re-identification. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR), pp 1278–1287

  42. Zhao H, Tian M, Sun S, Shao J, Yan J, Yi S, Wang X, Tang X (2017) Spindle net: Person re-identification with human body region guided feature decomposition and fusion. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 907–915

  43. Zhao R, Ouyang W, Wang X (2013) Person re-identification by salience matching. In: Proceedings of the IEEE international conference on computer vision (ICCV), pp 2528–2535

  44. Zheng L, Shen L, Tian L, Wang S, Wang J, Tian Q (2015) Scalable person re-identification: A benchmark. In: Proc. IEEE international conference on computer vision (ICCV)

  45. Zheng L, Yang Y, Hauptmann AG (2016) Person reidentification: Past, present and future. arXiv:1610.02984

  46. Zheng Z, Zheng L, Yang Y (2016) A discriminatively learned cnn embedding for person re-identification. arXiv:1611.05666

  47. Zheng Z, Zheng L, Yang Y (2017) Unlabeled samples generated by gan improve the person re-identification baseline in vitro. In: Proceedings of the IEEE international conference on computer vision (ICCV)

Download references

Acknowledgements

The authors are grateful to Mr. Jun Sun in Xidian University for his contributions to various experiments and collected data.

Author information

Authors and Affiliations

  1. School of Electronic Engineering, Xidian University, 2 South Taibai Road, Xian, Shaanxi, 710071, China

    Zhangping He, Cheolkon Jung, Qingtao Fu & Zhendong Zhang

Authors
  1. Zhangping He
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Cheolkon Jung
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Qingtao Fu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Zhendong Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Cheolkon Jung.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

An earlier version of this paper was presented in the 2018 IEEE Conference on Acoustics, Speech, and Signal Processing (ICASSP), Calgary, Alberta, Canada, April 15-20, 2018 [13]. This work was supported by the National Natural Science Foundation of China (No. 61271298) and the International S, T Cooperation Program of China (No. 2014DFG12780).

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

He, Z., Jung, C., Fu, Q. et al. Deep feature embedding learning for person re-identification based on lifted structured loss. Multimed Tools Appl 78, 5863–5880 (2019). https://doi.org/10.1007/s11042-018-6408-4

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-018-6408-4

Keywords

Search

Navigation