Skip to main content
SpringerLink
Log in

Integrating appearance features and soft biometrics for person re-identification

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Matching people in different camera views, commonly referred to as person re-identification, is an inherently challenging task due to the appearance disparity caused by view change.Other factors such as low image resolution and occlusion further compound this problem. As a highly demanded technique, person re-identification has been actively studied in recent years. Most of the existing approaches either focus on feature design or distance metric learning, based on appearance features. However, due to the view change, the appearance features may significantly vary for the same subject, resulting in matching difficulties. Instead of using features from a single modality, i.e, appearance, we propose to use multimodal features to improve the re-identification accuracy. Specifically, in this work, we leverage both appearance features and soft biometrics, i.e, human characteristics such as gender, to match individuals across cameras. We build multiple graphs, each of which represent one feature modality, and the graphs are then combined and optimized to derive the similarities between a probe and the gallery subjects. The proposed method is evaluated on the VIPeR dataset with annotated soft biometric labels. The results suggest that using multimodal features, e.g, appearance and soft biometrics, can improve the matching accuracy as compared to using appearance features only, and superior performance is obtained as compared to other state-of-the-art approaches.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

References

  1. Ahmed E, Jones M, Marks T (2015) An improved deep learning architecture for person re-identification. In: IEEE Conference on computer vision and pattern recognition (CVPR), pp 3908–3916

  2. An L, Yang S, Bhanu B (2015) Person re-identification by robust canonical correlation analysis. IEEE Signal Processing Letters 22(8):1103–1107

    Article  Google Scholar 

  3. An L, Chen X, Kafai M, Yang S, Bhanu B (2013) Improving person re-identification by soft biometrics based reranking. In: ACM/IEEE International conference on distributed smart cameras (ICDSC), pp 1–6

  4. An L, Kafai M, Yang S, Bhanu B (2015) Person re-identification with reference descriptor. IEEE Transactions on Circuits and Systems for Video Technology PP(99):1–1

    Google Scholar 

  5. Bedagkar-Gala A, Shah SK (2014) A survey of approaches and trends in person re-identification. Image Vis Comput 32(4):270–286

    Article  Google Scholar 

  6. Chen SZ, Guo CC, Lai JH (2016) Deep ranking for person re-identification via joint representation learning. IEEE Trans Image Process 25(5):2353–2367

    Article  MathSciNet  Google Scholar 

  7. Cheng DS, Cristan M, Stoppa M, Bazzani L, Murino V (2011) Custom pictorial structures for re-identification. In: British machine vision conference (BMVC), pp 68.1–68.11

  8. Gao Y, Wang M, Ji R, Wu X, Dai q (2014) 3-d object retrieval with hausdorff distance learning. IEEE Trans Ind Electron 61(4):2088–2098

    Article  Google Scholar 

  9. Gao Y, Wang M, Tao D, Ji R, Dai q (2012) 3-d object retrieval and recognition with hypergraph analysis. IEEE Trans Image Process 21(9):4290–4303

    Article  MathSciNet  Google Scholar 

  10. Gao Y, Wang M, Zha ZJ, Shen J, Li X, Wu X (2013) Visual-textual joint relevance learning for tag-based social image search. IEEE Trans Image Process 22(1):363–376

    Article  MathSciNet  Google Scholar 

  11. Gray D, Brennan S, Tao H (2007) Evaluating appearance models for recognition, reacquisition, and tracking. In: IEEE International workshop on performance evaluation of tracking and surveillance (PETS), pp 1–7

  12. Hirzer M, Roth PM, Köstinger M, Bischof H (2012) Relaxed pairwise learned metric for person re-identification. In: European conference on computer vision (ECCV), pp 780–793

  13. Huang Y, Liu Q, Zhang S, Metaxas D (2010) Image retrieval via probabilistic hypergraph ranking. In: IEEE Conference on computer vision and pattern recognition (CVPR), pp 3376–3383

  14. Ji R, Gao Y, Hong R, Liu Q, Tao D, Li X (2014) Spectral-spatial constraint hyperspectral image classification. IEEE Trans Geosci Remote Sens 52(3):1811–1824

    Article  Google Scholar 

  15. Jiang J, Hu R, Wang Z, Han Z (2014) Noise robust face hallucination via locality-constrained representation. IEEE Transactions on Multimedia 16(5):1268–1281

    Article  Google Scholar 

  16. Köstinger M, Hirzer M, Wohlhart P, Roth P, Bischof H (2012) Large scale metric learning from equivalence constraints. In: IEEE Conference on computer vision and pattern recognition (CVPR), pp 2288–2295

  17. Kuo CH, Khamis S, Shet V (2013) Person re-identification using semantic color names and rankboost. In: IEEE Workshop on applications of computer vision (WACV), pp 281–287

  18. Kviatkovsky I, Adam A, Rivlin E (2013) Color invariants for person reidentification. IEEE Transactions on Pattern Analysis and Machine Intelligence 35 (7):1622–1634

    Article  Google Scholar 

  19. Layne R, Hospedales T, Gong S (2012) Person re-identification by attributes. In: Proceedings of the british machine vision conference, pp 24.1–24.11

  20. Li Z, Chang S, Liang F, Huang T, Cao L, Smith J (2013) Learning locally-adaptive decision functions for person verification IEEE Conf Comput Vis Pattern Recognit (CVPR), pp 3610–3617

    Google Scholar 

  21. Liao S, Hu Y, Zhu X, Li SZ (2015) Person re-identification by local maximal occurrence representation and metric learning. In: IEEE Conf Comput Vis Pattern Recognit, pp 2197–2206

  22. Liu C, Gong S, Loy C, Lin X (2012) Person re-identification: What features are important?. In: European conference on computer vision workshops and demonstrations, pp 391–401

  23. Liu M, Zhang D (2016) Pairwise constraint-guided sparse learning for feature selection. IEEE Transactions on Cybernetics 46(1):298–310

    Article  MathSciNet  Google Scholar 

  24. Liu M, Zhang D, Chen S (2014) Attribute relation learning for zero-shot classification. Neurocomputing 139:34–46

    Article  Google Scholar 

  25. Liu M, Zhang D, Chen S, Xue H (2016) Joint binary classifier learning for ecoc-based multi-class classification. IEEE Transactions on Pattern Analysis and Machine Intelligence 38(11):2335–2341

  26. Liu X, Song M, Tao D, Zhou X, Chen C, Bu J (2014) Semi-supervised coupled dictionary learning for person re-identification. In: IEEE Conf Comput Vis Pattern Recognit (CVPR), pp 3550–3557

  27. Ma B, Su Y, Jurie F (2014) Covariance descriptor based on bio-inspired features for person re-identification and face verification. Image and Vision Computing 32(6-7):379–390

    Article  Google Scholar 

  28. Nixon MS, Correia PL, Nasrollahi K, Moeslund TB, Hadid A, Tistarelli M (2015) On soft biometrics. Pattern Recognition Letters 68, Part 2:218–230

    Article  Google Scholar 

  29. Paisitkriangkrai S, Shen C, van den Hengel A (2015) Learning to rank in person re-identification with metric ensembles. In: IEEE Conf Comput Vis Pattern Recognit (CVPR), pp 1846–1855

  30. Pedagadi S, Orwell J, Velastin S, Boghossian B (2013) Local fisher discriminant analysis for pedestrian re-identification. In: IEEE Conf Comput Vis Pattern Recognit (CVPR), pp 3318–3325

  31. Tao D, Guo Y, Song M, Li Y, Yu Z, Tang YY (2016) Person re-identification by dual-regularized KISS metric learning. IEEE Trans Image Process 25(6):2726–2738

    Article  MathSciNet  Google Scholar 

  32. Tao D, Jin L, Wang Y, Yuan Y, Li X (2013) Person re-identification by regularized smoothing KISS metric learning. IEEE Transactions on Circuits and Systems for Video Technology 23 (10):1675– -1685

    Article  Google Scholar 

  33. Vezzani R, Baltieri D, Cucchiara R (2013) People re-identification in surveillance and forensics: a survey. ACM Computing Surveys 46(2):29:1–293

    Article  Google Scholar 

  34. Wang F, Zuo W, Lin L, Zhang D, Zhang L (2016) Joint learning of single-image and cross-image representations for person re-identification. In: IEEE Conf Comput Vis Pattern Recognit (CVPR), pp 1288–1296

  35. Wang Z, Hu R, Liang C, Yu Y, Jiang J, Ye M, Chen J, Leng Q (2016) Zero-shot person re-identification via cross-view consistency. IEEE Transactions on Multimedia 18(2):260–272

    Article  Google Scholar 

  36. Wei H, Viallon M, Delattre BMA, Wang L, Pai VM, Wen H, Xue H, Guetter C, Croisille P, Zhu Y (2013) Assessment of cardiac motion effects on the fiber architecture of the human heart in vivo. IEEE Trans Med Imaging 32(10):1928–1938

    Article  Google Scholar 

  37. Xiong F, Gou M, Camps O, Sznaier M (2014) Person re-identification using kernel-based metric learning methods. In: European conference on computer vision, pp 1–16

  38. Xu D, Zheng H (2013) Person re-identification by multi-resolution saliency-weighted color histograms and local structural sparse coding. In: 7th international conference on image and graphics (ICIG), pp 477–482

  39. Yang Y, Yang J, Yan J, Liao S, Yi D, Li S (2014) Salient color names for person re-identification. In: European conference on computer vision (ECCV), pp 536–551

  40. Zhang J, Liang J, Zhao H (2013) Local energy pattern for texture classification using self-adaptive quantization thresholds. IEEE Trans Image Process 22(1):31–42

    Article  MathSciNet  Google Scholar 

  41. Zhang J, Zhao H, Liang J (2013) Continuous rotation invariant local descriptors for texton dictionary-based texture classification. Computer Vision and Image Understanding 117(1):56–75

    Article  Google Scholar 

  42. Zhao R, Ouyang W, Wang X (2013) Person re-identification by salience matching. In: IEEE International conference on computer vision (ICCV), pp 2528–2535

  43. Zhao R, Ouyang W, Wang X (2014) Learning mid-level filters for person re-identification. In: IEEE Conf Comput Vis Pattern Recognit (CVPR), pp 144–151

  44. Zheng WS, Gong S, Xiang T (2013) Reidentification by relative distance comparison. IEEE Transactions on Pattern Analysis and Machine Intelligence 35 (3):653–668

    Article  Google Scholar 

  45. Zheng WS, Gong S, Xiang T (2016) Towards open-world person re-identification by one-shot group-based verification. IEEE Transactions on Pattern Analysis and Machine Intelligence 38(3):591– 606

    Article  Google Scholar 

  46. Zheng WS, Li X, Xiang T, Liao S, Lai J, Gong S (2015) Partial person re-identification IEEE International conference on computer vision (ICCV), pp 4678–4686

    Google Scholar 

Download references

Acknowledgments

This work was supported in part by the Natural Science Foundation of China under Grant Numbers 61602193, 61501312, 61403265, and in part by the Fundamental Research Funds for the Central Universities (HUST: 2016YXMS063). This work was also supported in part by the Science and Technology Plan of Sichuan Province under Grant Number 2015SZ0226.

Author information

Authors and Affiliations

  1. National Key Laboratory of Science and Technology on Multi-spectral Information Processing, School of Automation, Huazhong University of Science and Technology, Wuhan, 430074, China

    Le An

  2. Department of Computer Science and Engineering, University of California, Riverside, CA, 92521, USA

    Xiaojing Chen

  3. School of International Business Communications, Dongbei University of Finance & Economics, Dalian, 116025, China

    Shuang Liu

  4. College of Electronics and Information Engineering, Sichuan University, Chengdu, 610064, China

    Yinjie Lei & Songfan Yang

Authors
  1. Le An
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xiaojing Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Shuang Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yinjie Lei
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Songfan Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Xiaojing Chen.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

An, L., Chen, X., Liu, S. et al. Integrating appearance features and soft biometrics for person re-identification. Multimed Tools Appl 76, 12117–12131 (2017). https://doi.org/10.1007/s11042-016-4070-2

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-016-4070-2

Keywords

Search

Navigation