Skip to main content
SpringerLink
Log in

Simultaneous visual-appearance-level and spatial-temporal-level dictionary learning for video-based person re-identification

  • Original Article
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

Person re-identification (re-id) plays an important role in video surveillance and forensics applications. In many cases, person re-id should be conducted between video clips, i.e., given a query pedestrian video from one camera, the re-id system should retrieve the video clips containing the same person from other cameras. However, person re-id between videos, which we call video-based person re-id, has not been well studied. In this paper, we propose a visual-appearance-level and spatial-temporal-level dictionary learning (VSDL) approach for video-based person re-id. Specifically, we first employ two kinds of models to represent each walking cycle in the video, i.e., visual-appearance features of all frames within the walking cycle, and a spatial-temporal feature vector. By separately learning a visual-appearance-level dictionary and a spatial-temporal-level dictionary from two kinds of representations, each walking cycle can be represented as a coding coefficient. To enhance the discriminative ability of the obtained coding coefficients, we design a representation coefficient discriminant term for VSDL. Experiments on the public iLIDS-VID and PRID 2011 datasets demonstrate the effectiveness of VSDL.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

References

  1. Ahmed E, Jones M, Marks TK (2015) An improved deep learning architecture for person re-identification. In: IEEE conference on CVPR, pp 3908–3916

  2. Baltieri D, Vezzani R, Cucchiara R (2013) Learning articulated body models for people re-identification. In: ACM conference on multimedia, pp 557–560

  3. Chen D, Yuan Z, Hua G, Zheng N, Wang J (2015) Similarity learning on an explicit polynomial kernel feature map for person re-identification. In: IEEE conference on CVPR

  4. Chen J, Wang Y, Tang YY (2015) Person re-identification by exploiting spatio-temporal cues and multi-view metric learning. IEEE Signal Process Lett 23(7):998–1002

    Article  Google Scholar 

  5. Farenzena M, Bazzani L, Perina A, Murino V, Cristani M (2010) Person re-identification by symmetry-driven accumulation of local features. In: IEEE conference on CVPR, pp 2360–2367

  6. Gangeh MJ, Ghodsi A, Kamel MS (2013) Kernelized supervised dictionary learning. IEEE Trans Signal Process 61(19):4753–4767

    Article  MathSciNet  Google Scholar 

  7. Gu S, Zhang L, Zuo W, Feng X (2014) Projective dictionary pair learning for pattern classification. In: NIPS, pp 793–801

  8. Guillaumin M, Verbeek J, Schmid C (2009) Is that you? Metric learning approaches for face identification. In: IEEE conference on ICCV, pp 498–505

  9. Hirzer M, Beleznai C, Roth PM, Bischof H (2011) Person re-identification by descriptive and discriminative classification. In: Proceedings of 17th Scandinavian conference on image analysis (SCIA) 2011, Ystad, Sweden, pp 91–102

  10. Hirzer M, Roth PM, Bischof H (2012) Person re-identification by efficient impostor-based metric learning. In: IEEE conference on AVSS, pp 203–208

  11. Hirzer M, Roth PM, Köstinger M, Bischof H (2012) Relaxed pairwise learned metric for person re-identification. In: ECCV, pp 780–793

  12. Huang DA, Wang YCF (2013) Coupled dictionary and feature space learning with applications to cross-domain image synthesis and recognition. In: IEEE conference on ICCV, pp 2496–2503

  13. Jiang Z, Lin Z, Davis LS (2013) Label consistent k-svd: learning a discriminative dictionary for recognition. IEEE Trans Pattern Anal Mach Intell 35(11):2651–2664

    Article  Google Scholar 

  14. Jing XY, Zhu X, Wu F, You X, Liu Q, Yue D, Hu R, Xu B (2015) Super-resolution person re-identification with semi-coupled low-rank discriminant dictionary learning. In: IEEE conference on CVPR, pp 695–704

  15. Kostinger M, Hirzer M, Wohlhart P, Roth PM, Bischof H (2012) Large scale metric learning from equivalence constraints. In: IEEE conference on CVPR, pp 2288–2295

  16. Li A, Liu L, Wang K, Liu S, Yan S (2015) Clothing attributes assisted person re-identification. IEEE Trans Circuits Syst Video Technol 25(5):869–878

    Article  Google Scholar 

  17. Li S, Shao M, Fu Y (2015) Cross-view projective dictionary learning for person re-identification. In: IJCAI, pp 2155–2161

  18. Li W, Wang X (2013) Locally aligned feature transforms across views. In: IEEE conference on CVPR, pp 3594–3601

  19. Lisanti G, Masi I, Bagdanov A, Del Bimbo A (2015) Person re-identification by iterative re-weighted sparse ranking. IEEE Trans Pattern Anal Mach Intell 37(8):1629–1642

    Article  Google Scholar 

  20. Liu C, Loy CC, Gong S, Wang G (2013) Pop: person re-identification post-rank optimisation. In: IEEE conference on ICCV, pp 441–448

  21. Liu H, Ma B, Qin L, Pang J, Zhang C, Huang Q (2015) Set-label modeling and deep metric learning on person re-identification. Neurocomputing 151:1283–1292

    Article  Google Scholar 

  22. Liu K, Ma B, Zhang W, Huang R (2015) A spatio-temporal appearance representation for viceo-based pedestrian re-identification. In: ICCV, pp 3810–3818

  23. Liu X, Song M, Tao D, Zhou X, Chen C, Bu J (2014) Semi-supervised coupled dictionary learning for person re-identification. In: IEEE conference on CVPR, pp 3550–3557

  24. Ma AJ, Yuen PC, Li J (2013) Domain transfer support vector ranking for person re-identification without target camera label information. In: IEEE conference on ICCV, pp 3567–3574

  25. Mignon A, Jurie F (2012) Pcca: A new approach for distance learning from sparse pairwise constraints. In: IEEE conference on CVPR, pp 2666–2672

  26. Pedagadi S, Orwell J, Velastin S, Boghossian B (2013) Local fisher discriminant analysis for pedestrian re-identification. In: IEEE conference on CVPR, pp 3318–3325

  27. Prosser B, Zheng WS, Gong S, Xiang T, Mary Q (2010) Person re-identification by support vector ranking. In: BMVC, pp 1–11

  28. Qiu Q, Ni J, Chellappa R (2014) Dictionary-based domain adaptation methods for the re-identification of faces. In: Person re-Identification. Springer, London, pp 269–285

  29. Tao D, Jin L, Wang Y, Yuan Y, Li X (2013) Person re-identification by regularized smoothing kiss metric learning. IEEE Trans Circuits Syst Video Technol 23(10):1675–1685

    Article  Google Scholar 

  30. Wang T, Gong S, Zhu X, Wang S (2014) Person re-identification by video ranking. In: ECCV, pp 688–703

  31. Wang T, Gong S, Zhu X, Wang S (2016) Person re-identification by discriminative selection in video ranking. IEEE Trans Pattern Anal Mach Intell. https://doi.org/10.1109/TPAMI.2016.2522418

    Article  Google Scholar 

  32. Wang Z, Hu R, Liang C, Jiang J, Sun K, Leng Q, Huang B (2015) Person re-identification using data-driven metric adaptation. In: MultiMedia modeling—21st international conference, MMM 2015, Sydney, NSW, Australia, 5–7 Jan 2015, proceedings, part II, pp 195–207

  33. Wang Z, Hu R, Liang C, Yu Y, Jiang J, Ye M, Chen J, Leng Q (2016) Zero-shot person re-identification via cross-view consistency. IEEE Trans Multimed 18(2):260–272

    Article  Google Scholar 

  34. Wang Z, Hu R, Liang C, Yu Y, Jiang J, Ye M, Chen J, Leng Q (2017) Noise robust face image super-resolution through smooth sparse representation. IEEE Trans Cybern. https://doi.org/10.1109/TCYB.2016.2594184

    Article  Google Scholar 

  35. Wang Z, Hu R, Yu Y, Jiang J, Liang C, Wang J (2016) Scale-adaptive low-resolution person re-identification via learning a discriminating surface. In: Proceedings of the twenty-fifth international joint conference on artificial intelligence, IJCAI 2016, New York, NY, 9–15 July 2016, pp 2669–2675

  36. Weinberger K, Saul L (2009) Distance metric learning for large margin nearest neighbor classification. J Mach Learn Res 10:207–244

    MATH  Google Scholar 

  37. Wu Z, Li Y, Radke RJ (2015) Viewpoint invariant human re-identification in camera networks using pose priors and subject-discriminative features. IEEE Trans Pattern Anal Mach Intell 37(5):1095–1108

    Article  Google Scholar 

  38. Yang J, Wright J, Huang TS, Ma Y (2010) Image super-resolution via sparse representation. IEEE Trans Image Process 19(11):2861–2873

    Article  MathSciNet  Google Scholar 

  39. Yang M, Zhang D, Feng X (2016) Fisher discrimination dictionary learning for sparse representation. In: IEEE conference on ICCV, pp 543–550

  40. Yang M, Zhang L, Feng X, Zhang D (2011) Fisher discrimination dictionary learning for sparse representation. In: ICCV, pp 543–550

  41. Yang Y, Yang J, Yan J, Liao S, Yi D, Li SZ (2014) Salient color names for person re-identification. In: ECCV, pp 536–551

  42. You J, Wu A, Li X, Zheng WS (2016) Top-push video-based person re-identification. arXiv preprint arXiv:1604.08683

  43. Zhang H, Cao X, Ho JKL, Chow TWS (2017) Object-level video advertising: an optimization framework. IEEE Trans Ind Inform 13(2):520–531

    Article  Google Scholar 

  44. Zhao R, Ouyang W, Wang X (2013) Person re-identification by salience matching. In: IEEE conference on ICCV, pp 2528–2535

  45. Zhao R, Ouyang W, Wang X (2013) Unsupervised salience learning for person re-identification. In: IEEE conference on CVPR, pp 3586–3593

  46. Zhao R, Ouyang W, Wang X (2014) Learning mid-level filters for person re-identification. In: IEEE conference on CVPR, pp 144–151

  47. Zheng L, Shen L, Tian L, Wang S, Wang J, Bu J, Tian Q (2015) Scalable person re-identification: a benchmark. In: IEEE conference on ICCV, pp 1116–1124

  48. Zheng WS, Gong S, Xiang T (2013) Reidentification by relative distance comparison. IEEE Trans Pattern Anal Mach Intell 35(3):653–668

    Article  Google Scholar 

  49. Zhu X, Jing XY, Wu F, Feng H (2016) Video-based person re-identification by simultaneously learning intra-video and inter-video distance metrics. In: IJCAI, pp 3552–3559

  50. Zhuang Y, Wang YF, Wu F, Zhang Y, Lu W (2013) Supervised coupled dictionary learning with group structures for multi-modal retrieval. In: AAAI, pp 1070–1076

Download references

Acknowledgements

The authors would like to thank the editors and anonymous reviewers for their constructive comments and suggestions. This work was supported by the National Key Research and Development Program of China under Grant No. 2017YFB0202001, NSFC Key Project of General Technology Fundamental Research United Fund No. U1736211, the National Nature Science Foundation of China under Grant Nos. 61672208, U1504611, 41571417, the Science and Technique Development Program of Henan under Grant Nos. 172102210186, 182102311066, the Medical Education Research Project of Henan No. Wjlx2016095.

Author information

Authors and Affiliations

  1. School of Computer and Information Engineering, Henan University, Kaifeng, China

    Xiaoke Zhu

  2. School of Computer, Wuhan University, Wuhan, China

    Xiaoke Zhu, Xiao-Yuan Jing, Fei Ma & Li Cheng

  3. School of computer, Beijing University of Posts and Telecommunications, Beijing, China

    Yilin Ren

Authors
  1. Xiaoke Zhu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xiao-Yuan Jing
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Fei Ma
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Li Cheng
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Yilin Ren
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Xiaoke Zhu.

Ethics declarations

Conflict of interest

The authors declare that there is no conflict of interests regarding the publication of this paper.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhu, X., Jing, XY., Ma, F. et al. Simultaneous visual-appearance-level and spatial-temporal-level dictionary learning for video-based person re-identification. Neural Comput & Applic 31, 7303–7315 (2019). https://doi.org/10.1007/s00521-018-3529-7

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-018-3529-7

Keywords

Search

Navigation