Skip to main content
SpringerLink
Log in

Tensor-based sparse canonical correlation analysis via low rank matrix approximation for RGB-D long-term person re-identification

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Person re-identification can be a part of almost any multi-camera surveillance systems. Most previous works propose strategies for short-term person re-identification which are usually driven from appearance features of RGB images. However, when people appear in excessive lighting or change clothes (i.e. long-term case), short-term person re-identification approaches have a tendency to fail. In this paper, we propose a novel approach for long-term person re-identification by employing depth videos of RGB-D sensors. We also develop a sparse canonical correlation analysis using a local third-order tensor model to accomplish multi-level person re-identification. The tensor representations of images make the space for performing the multi-level person re-identification simpler compared to existing methods. Finally, we evaluate our experiments on RGB-D long-term datasets consisting of BIWI RGBD-ID dataset and IAS-Lab RGBD-ID dataset. The results demonstrate the efficiency of the proposed method compared to recent methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

References

  1. Ahmed E, Jones M, Marks TK (2015) An improved deep learning architecture for person re-identification. In: Proceedings of the IEEE international conference on computer vision and pattern recognition

  2. An L, Yang S, Bhanu B (2015) Person re-identification by robust canonical correlation analysis. IEEE Signal Process Lett 22(8):1103–1107

    Article  Google Scholar 

  3. An L, Kafai M, Yang S, Bhanu B (2016) Person re-identification with reference descriptor. IEEE Trans Circuits Syst Video Technol 26(4):776–787

    Article  Google Scholar 

  4. An L, Chen X, Yang S, Bhanu B (2016) Sparse representation matching for person re-identification. Inf Sci 355:74–89

    Article  Google Scholar 

  5. An L, Qin Z, Chen X, Yang S (2017) Multi-level common space learning for person re-identification. IEEE Trans Circuits Syst Video Technol. https://doi.org/10.1109/TCSVT.2017.2680118

  6. Barbosa BI, Cristani M, Bue AD, Bazzani L, Murino V (2012) Re-identification with rgb-d sensors. In: First international workshop on re-identification. Springer, Berlin/Heidelberg, pp 433–442

    Google Scholar 

  7. Chahla C, Snoussi H, Abdallah F, Dornaika F (2017) Discriminant quaternion local binary pattern embedding for person reidentification through prototype formation and color categorization. Eng Appl Artif Intell 58:27–33

    Article  Google Scholar 

  8. Chattopadhyay P, Sural S, Mukherjee J (2014) Frontal gait recognition from incomplete sequences using RGB-D camera. IEEE Trans Inf Forensics Secur 9(11):1843–1856

    Article  Google Scholar 

  9. Chu H, Qi M, Liu H, Jiang J (2017) Local region partition for person re-identification. Multimed Tools Appl 78:27067–27083. https://doi.org/10.1007/s11042-017-4817-4

    Article  Google Scholar 

  10. Dai J, Zhang Y, Lu H, Wang H (2018) Cross-view semantic projection learning for person re-identification. Pattern Recogn 75:63–76

    Article  Google Scholar 

  11. Fang W, Hu HM, Hu Z, Liao S, Li B (2018) Perceptual hash-based feature description for person re-identification. Neurocomputing 272:520–531

    Article  Google Scholar 

  12. Farenzena M, Bazzani L, Perina A, Murino V, Cristani M (2013) Symmetry-driven accumulation of local features for human characterization and re-identification. Comput Vis Image Underst 117(2):130–144

    Article  Google Scholar 

  13. Fehr D, Cherian A, Sivalingam R, Nickolay S, Morellas V, Papanikolopoulos N (2012) Compact covariance descriptors in 3d point clouds for object recognition. In: Proceedings of the IEEE international conference on robotics and automation (ICRA), pp 1793–1798

  14. Figueira D, Bazzani L, Minh HQ, Cristani M, Bernardino A, Murino V (2013) Semi-supervised multi-feature learning for person re-identification. In: Proceedings of IEEE international conference on advanced video and signal based surveillance (AVSS), pp 111–116

  15. Gao B, Zeng M, Xu S, Sun F, Guo J (2016) Person re-identification with discriminatively trained viewpoint invariant orthogonal dictionaries. Electron Lett 52(23):1914–1916

    Article  Google Scholar 

  16. García J, Martinel N, Gardel A, Bravo I, Foresti GL, Micheloni C (2016) Modeling feature distances by orientation driven classifiers for person re-identification. J Vis Commun Image Represent 38:115–129

    Article  MATH  Google Scholar 

  17. Geng Y, Hu HM, Zeng G, Zheng J (2015) A person re-identification algorithm by exploiting region-based feature salience. J Vis Commun Image Represent 29:89–102

    Article  Google Scholar 

  18. Hu HM, Fang W, Zeng G, Hu Z, Li B (2017) A person re-identification algorithm based on pyramid color topology feature. Multimed Tools Appl 76(24):26633–26646

    Article  Google Scholar 

  19. Imani Z, Soltanizadeh H (2016) Person reidentification using local pattern descriptors and anthropometric measures from videos of kinect sensor. IEEE Sensors J 16(16):6227–6238

    Article  Google Scholar 

  20. Imani Z, Soltanizadeh H (2018) Histogram of the node strength and histogram of the edge weight: two new features for RGB-D person re-identification. Sci China Inf Sci 61(9):092108. https://doi.org/10.1007/s11432-016-9086-8

    Article  Google Scholar 

  21. Imani Z, Soltanizadeh H (2019) Local binary pattern, local derivative pattern and skeleton features for RGB-D person re-identification. Natl Acad Sci Lett 42(3):233–238

    Article  Google Scholar 

  22. Imani Z, Soltanizadeh H, & Orouji AA (2019) Short-Term Person Re-identification Using RGB, Depth and Skeleton Information of RGB-D Sensors. Iran J Sci Technol Trans Electr Eng https://doi.org/10.1007/s40998-019-00249-9

  23. Khedher MI, El-Yacoubi MA, Dorizzi B (2017) Fusion of appearance and motion-based sparse representations for multi-shot person re-identification. Neurocomputing 248:94–104

    Article  Google Scholar 

  24. Kolda TG, Bader BW (2009) Tensor decompositions and applications. SIAM Rev 51(3):455–500

    Article  MathSciNet  MATH  Google Scholar 

  25. Köstinger M, Hirzer M, Wohlhart P, Roth P, Bischof H (2012) Large scale metric learning from equivalence constraints. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR), pp 2288–2295

  26. Koukiou G, Anastassopoulos V (2018) Local difference patterns for drunk person identification. Multimed Tools Appl 77(8):9293–9305

    Article  Google Scholar 

  27. Kviatkovsky I, Adam A, Rivlin E (2013) Color invariants for person reidentification. IEEE Trans Pattern Anal Mach Intell 35(7):1622–1634

    Article  Google Scholar 

  28. Li W, Wu Y, Li J (2017) Re-identification by neighborhood structure metric learning. Pattern Recogn 61:327–338

    Article  Google Scholar 

  29. Liao S, Hu Y, Zhu X, Li SZ (2015) Person re-identification by local maximal occurrence representation and metric learning. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR), pp 2197–2206

  30. Liong VE, Lu J, Ge Y (2015) Regularized local metric learning for person re-identification. Pattern Recogn Lett 68:288–296

    Article  Google Scholar 

  31. Liu Y, Lasang P, Siegel M, Sun Q (2015) Geodesic invariant feature: a local descriptor in depth. IEEE Trans Image Process 24(1):236–248

    Article  MathSciNet  MATH  Google Scholar 

  32. Liu H, Hu L, Ma L (2017) Online RGB-D person re-identification based on metric model update. CAAI Trans Intell Technol 2(1):48–55

    Article  Google Scholar 

  33. Lu H, Plataniotis KN, Venetsanopoulos AN (2011) A survey of multilinear subspace learning for tensor data. Pattern Recogn 44:1540–1551

    Article  MATH  Google Scholar 

  34. Ma B, Su Y, Jurie F (2014) Covariance descriptor based on bio-inspired features for person re-identification and face verification. Image Vis Comput 32(6):379–390

    Article  Google Scholar 

  35. Marcolin F, Vezzetti E (2017) Novel descriptors for geometrical 3D face analysis. Multimed Tools Appl 76(12):13805–13834

    Article  Google Scholar 

  36. Martinel N, Das A, Micheloni C, Chowdhury AR (2015) Re-identification in the function space of feature warps. IEEE Trans Pattern Anal Mach Intell 37(8):1656–1669

    Article  Google Scholar 

  37. Mignon A, Jurie F (2012) PCCA: a new approach for distance learning from sparse pairwise constraints. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR), pp 2666–2672

  38. Munaro M, Fossati A, Basso A, Menegatti E, Gool LV (2014) One-shot person re-identification with a consumer depth camera. In: Advances in computer vision and pattern recognition. Springer, London, pp 161–181

    Google Scholar 

  39. Munaro M, Ghidoni S, Dizmen DT, Menegatti E (2014) A feature-based approach to people re-identification using skeleton keypoints. In: Proceedings of the IEEE international conference on robotics and automation (ICRA), pp 5644–5651

  40. Nanni L, Munaro M, Ghidoni S, Menegatti E, Brahnam S (2016) Ensemble of different approaches for a reliable person re-identification system. Appl Comput Inform 12(2):142–153

    Article  Google Scholar 

  41. Pala F, Satta R, Fumera G, Roli F (2016) Multi-modal person re-identification using RGB-D cameras. IEEE Trans Circuits Syst Video Technol 26(4):788–799

    Article  Google Scholar 

  42. Patruno C, Marani R, Cicirelli G, Stella E, D'Orazio T (2019) People re-identification using skeleton standard posture and color descriptors from RGB-D data. Pattern Recogn 89:77–90

    Article  Google Scholar 

  43. Pedagadi S, Orwell J, Velastin S, Boghossian B (2013) Local Fisher discriminant analysis for pedestrian re-identification. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR), pp 3318–3325

  44. Prosser B, Zheng WS, Gong S, Xiang T (2010) Person reidentification by support vector ranking. In: Proceedings of the British machine vision conference (BMVC), pp 21.1–21.11

  45. Rahmawati E, Listyasari M, Aziz AS, Sukaridhoto S, Damastuti FA, Bachtiar MM, Sudarsono A (2017) Digital signature on file using biometric fingerprint with fingerprint sensor on smartphone. In: International electronics symposium on engineering technology and applications (IES-ETA), pp 234–238

  46. Ren L, Lu J, Feng J, Zhou J (2017) Multi-modal uniform deep learning for RGB-D person re-identification. Pattern Recogn 72:446–457

    Article  Google Scholar 

  47. Skelly LJ, Sclaroff S (2007) Improved feature descriptors for 3d surface matching. In: Proceedings of the society of photo-optical instrumentation engineers (SPIE), pp 67 620A–67 620A

  48. Vezzetti E, Marcolin F, Stola V (2013) 3D human face soft tissues landmarking method: an advanced approach. Comput Ind 64(9):1326–1354

    Article  Google Scholar 

  49. Wang Z, Hu R, Liang C, Yu Y (2016) Zero-shot person re-identification via cross-view consistency. IEEE Trans Multimed 18(2):260–272

    Article  Google Scholar 

  50. Wang J, Wang Z, Liang C, Gao C, Sang N (2018) Equidistance constrained metric learning for person re-identification. Pattern Recogn 74:38–51

    Article  Google Scholar 

  51. Wu A, Zheng WS, Lai J (2017) Robust depth-based person re-identification. IEEE Trans Image Process 26(6):2588–2603

    Article  MathSciNet  MATH  Google Scholar 

  52. Xu X, Li W, Xu D (2015) Distance metric learning using privileged information for face verification and person re-identification. IEEE Trans Neural Netw Learn Syst 26(12):3150–3162

    Article  MathSciNet  Google Scholar 

  53. Xu Y, Guo J, Huang Z, Qiu W (2018) Sparse coding with cross-view invariant dictionaries for person re-identification. Multimed Tools Appl 77(9):10715–10732

    Article  Google Scholar 

  54. Yan J, Zheng W, Zhou X, Zhao Z (2012) Sparse 2-d canonical correlation analysis via low rank matrix approximation for feature extraction. IEEE Signal Process Lett 19(1):51–54

    Article  Google Scholar 

  55. Yan C, Luo M, Liu W, Zheng Q (2018) Robust dictionary learning with graph regularization for unsupervised person re-identification. Multimed Tools Appl 77(3):3553–3577

    Article  Google Scholar 

  56. Yang Z, Hu X, Dai F, Pang J, Jiang T, Tao D (2018) Person re-identification by discriminant analytical least squares metric learning. Mach Vis Appl 29:1019–1031. https://doi.org/10.1007/s00138-018-0917-z

    Article  Google Scholar 

  57. Zhao C, Wang X, Miao D, Wang H, Zheng W, Xu Y, Zhang D (2018) Maximal granularity structure and generalized multi-view discriminant analysis for person re-identification. Pattern Recogn 79:79–96

    Article  Google Scholar 

  58. Zhao C, Chen Y, Wang X, Wong WK, Miao D, Lei J (2018) Kernelized random KISS metric learning for person re-identification. Neurocomputing 275:403–417

    Article  Google Scholar 

  59. Zhao C, Chen Y, Wei Z, Miao D, Gu X (2018) QRKISS: a two-stage metric learning via QR-decomposition and KISS for person re-identification. Neural Process Lett 49:899–922. https://doi.org/10.1007/s11063-018-9820-x

    Article  Google Scholar 

  60. Zheng WS, Gong S, Xiang T (2013) Reidentification by relative distance comparison. IEEE Trans Pattern Anal Mach Intell 35(3):653–668

    Article  Google Scholar 

  61. Zhou Q, Zheng S, Ling H, Su H, Wu S (2017) Joint dictionary and metric learning for person re-identification. Pattern Recogn 72:196–206

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Electrical and Computer Engineering Faculty, Semnan University, Semnan, Iran

    Zeynab Imani, Hadi Soltanizadeh & Ali A. Orouji

Authors
  1. Zeynab Imani
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Hadi Soltanizadeh
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ali A. Orouji
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Hadi Soltanizadeh.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Imani, Z., Soltanizadeh, H. & Orouji, A.A. Tensor-based sparse canonical correlation analysis via low rank matrix approximation for RGB-D long-term person re-identification. Multimed Tools Appl 79, 11787–11811 (2020). https://doi.org/10.1007/s11042-019-08311-8

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-019-08311-8

Keywords

Search

Navigation