Skip to main content
SpringerLink
Log in

Two-stream person re-identification with multi-task deep neural networks

  • Special Issue Paper
  • Published:
Machine Vision and Applications Aims and scope Submit manuscript

Abstract

Person re-identification (re-id) with images is very useful in video surveillance to find specific targets. However, it is challenging due to the complex variations of human poses, camera viewpoints, lighting, occlusion, resolution, background clutter and so on. The key to tackle this problem is how to represent the body and match these representations among frames. Current methods usually use the features of the whole bodies, and the performance may be reduced because of part invisibility. To solve this problem, we propose a two-stream strategy to use parts and bodies simultaneously. It utilizes a multi-task learning framework with deep neural networks (DNNs). Part detection and body recognition are performed as two tasks, and the features are extracted by two DNNs. The features are connected to multi-task learning to compute the mapping model from features to identifications. With this model, re-id can be achieved. Experimental results on a challenging task show the effectiveness of the proposed method.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

References

  1. Argyriou, A., Evgeniou, T., Pontil, M.: Multi-task feature learning. In: Conference on Advances in Neural Information Processing Systems, pp. 41–48 (2007)

  2. Bengio, Y., et al.: Learning deep architectures for ai. Foundations and trends \(\textregistered \). Mach. Learn. 2(1), 1–127 (2009)

    Article  MATH  Google Scholar 

  3. Bromley, J., Guyon, I., Lecun, Y., Sackinger, E., Shah, R.: Signature verification using a "siamese" time delay neural network. In: International Conference on Neural Information Processing Systems, pp. 737–744 (1993)

  4. Caruana, R.: Multitask learning. Mach. Learn. 28(1), 41–75 (1997)

    Article  MathSciNet  Google Scholar 

  5. Chen, J., Liu, J., Ye, J.: Learning incoherent sparse and low-rank patterns from multiple tasks. ACM Trans. Knowl. Discov. Data 5(4), 22 (2012)

    Article  Google Scholar 

  6. Chen, X., Lin, Q., Kim, S., Carbonell, J.G., Xing, E.P.: Smoothing proximal gradient method for general structured sparse learning. In: Twenty-Seventh Conference on Uncertainty in Artificial Intelligence, pp. 105–114 (2011)

  7. Dalal, N., Triggs, B.: Histograms of oriented gradients for human detection. In: IEEE Conference on Computer Vision and Pattern Recognition, vol. 1, pp. 886–893. IEEE (2005)

  8. Evgeniou, T., Pontil, M.: Regularized multi-task learning. In: International Conference on Knowledge Discovery and Data Mining, pp. 109–117 (2004)

  9. Gong, P., Ye, J., Zhang, C.: Robust multi-task feature learning. In: International Conference on Knowledge Discovery & Data Mining, pp. 895–903 (2012)

  10. Hermans, A., Beyer, L., Leibe, B.: In defense of the triplet loss for person re-identification. arXiv preprint arXiv:1703.07737 (2017)

  11. Jalali, A., Sanghavi, S., Ruan, C., et al.: A dirty model for multi-task learning. In: Lafferty, J.D., Williams, C.K.I., Shawe-Taylor, J., Zemel, R.S., Culotta, A. (eds.) Advances in Neural Information Processing Systems, pp. 964–972. Curran Associates, Inc. (2010)

  12. Ji, S., Ye, J.: An accelerated gradient method for trace norm minimization. In: International Conference on Machine Learning, pp. 457–464 (2009)

  13. Jia, Y., Shelhamer, E., Donahue, J., Karayev, S., Long, J., Girshick, R., Guadarrama, S., Darrell, T.: Caffe: convolutional architecture for fast feature embedding. arXiv preprint arXiv:1408.5093 (2014)

  14. Kang, Z., Grauman, K., Sha, F.: Learning with whom to share in multi-task feature learning. In: International Conference on Machine Learning, pp. 521–528 (2011)

  15. Karpagavalli, P., Ramprasad, A.V.: An adaptive hybrid gmm for multiple human detection in crowd scenario. Multimed. Tools Appl. 76, 1–21 (2016)

    Google Scholar 

  16. Li, W., Zhao, R., Xiao, T., Wang, X.: Deepreid: deep filter pairing neural network for person re-identification. In: Computer Vision and Pattern Recognition, pp. 152–159 (2014)

  17. Li, X., Zhao, L., Wei, L., Yang, M.H., Wu, F., Zhuang, Y., Ling, H., Wang, J.: Deepsaliency: multi-task deep neural network model for salient object detection. IEEE Trans Image Process 25(8), 3919 (2016)

    Article  MathSciNet  Google Scholar 

  18. Lin, W., Shen, Y., Yan, J., Xu, M., Wu, J., Wang, J., Lu, K.: Learning correspondence structures for person re-identification. IEEE Trans Image Process 26(5), 2438–2453 (2017)

    Article  MathSciNet  Google Scholar 

  19. Liu, W., Yang, X., Tao, D., Cheng, J., Tang, Y.: Multiview dimension reduction via hessian multiset canonical correlations. Inf Fusion 41, 119–128 (2017)

    Article  Google Scholar 

  20. Mar, N.J., Vazquez, D., Lopez, A.M., Amores, J., Kuncheva, L.I.: Occlusion handling via random subspace classifiers for human detection. IEEE Trans. Cybern. 44(3), 342–354 (2017)

    Google Scholar 

  21. Miseikis, J., Borges, P.V.K.: Joint human detection from static and mobile cameras. IEEE Trans. Intell. Transp. Syst. 16(2), 1018–1029 (2015)

    Google Scholar 

  22. Sang, J., Xu, C., Liu, J.: User-aware image tag refinement via ternary semantic analysis. IEEE Trans. Multimed. 14(3), 883–895 (2012)

    Article  Google Scholar 

  23. Shao, L., Wu, D., Li, X.: Learning deep and wide: a spectral method for learning deep networks. IEEE Trans. Neural Netw. Learn. Syst. 25(12), 2303–2308 (2014)

    Article  Google Scholar 

  24. Srivastava, N., Hinton, G.E., Krizhevsky, A., Sutskever, I., Salakhutdinov, R.: Dropout: a simple way to prevent neural networks from overfitting. J. Mach. Learn. Res. 15(1), 1929–1958 (2014)

    MathSciNet  MATH  Google Scholar 

  25. Sun, C., Wang, D., Lu, H.: Person re-identification via distance metric learning with latent variables. IEEE Trans. Image Process. 26(1), 23–34 (2016)

    Article  MathSciNet  Google Scholar 

  26. Tibshirani, R.: Regression shrinkage and selection via the lasso: a retrospective. J. R. Stat. Soc. 73(3), 267–288 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  27. Varior, R.R., Wang, G., Lu, J., Liu, T.: Learning invariant color features for person reidentification. IEEE Trans. Image Process. 25(7), 3395–3410 (2016)

    Article  MathSciNet  Google Scholar 

  28. Xiao, F., Liu, W., Li, Z., Chen, L., Wang, R.: Noise-tolerant wireless sensor networks localization via multi-norms regularized matrix completion. IEEE Trans. Veh. Technol. PP(99), 1–1 (2017)

    Article  Google Scholar 

  29. Xiao, F., Wang, Z., Ye, N., Wang, R., Li, X.Y.: One more tag enables fine-grained rfid localization and tracking. IEEE ACM Trans. Netw. PP(99), 1–14 (2017)

    Google Scholar 

  30. Xiao, T., Li, H., Ouyang, W., Wang, X.: Learning deep feature representations with domain guided dropout for person re-identification. In: CVPR (2016)

  31. Xiao, T., Li, S., Wang, B., Lin, L., Wang, X.: Joint detection and identification feature learning for person search. In: CVPR (2017)

  32. Xu, C.: Exploiting social-mobile information for location visualization. ACM 8, 39 (2017)

    Google Scholar 

  33. Yan, Y., Ricci, E., Liu, G., Sebe, N.: Egocentric daily activity recognition via multitask clustering. IEEE Trans. Image Process. 24(10), 2984–2995 (2015)

    Article  MathSciNet  Google Scholar 

  34. Yan, Y., Ricci, E., Subramanian, R., Liu, G., Sebe, N.: Multitask linear discriminant analysis for view invariant action recognition. IEEE Trans. Image Process. 23(12), 5599–5611 (2014)

    Article  MathSciNet  MATH  Google Scholar 

  35. Yang, X., Liu, W., Tao, D., Cheng, J.: Canonical correlation analysis networks for two-view image recognition. Inf. Sci. 385(C), 338–352 (2017)

    Article  Google Scholar 

  36. Yi, D., Lei, Z., Liao, S., et al.: Deep metric learning for person re-identification. In: ICPR ’14 Proceedings of the 2014 22nd International Conference on Pattern Recognition, pp. 34–39. IEEE Computer Society, Washington, DC, USA (2014)

  37. Yogarajah, P., Chaurasia, P., Condell, J., Prasad, G.: Enhancing gait based person identification using joint sparsity model and -norm minimization. Inf. Sci. 308, 3–22 (2015)

    Article  Google Scholar 

  38. Yu, J., Rui, Y., Chen, B.: Exploiting click constraints and multi-view features for image re-ranking. IEEE Trans. Multimed. 16(1), 159–168 (2013)

    Article  Google Scholar 

  39. Yu, J., Rui, Y., Tao, D.: Click prediction for web image reranking using multimodal sparse coding. IEEE Trans. Image Process. 23(5), 2019–32 (2014)

    Article  MathSciNet  MATH  Google Scholar 

  40. Yu, J., Tao, D., Wang, M., Rui, Y.: Learning to rank using user clicks and visual features for image retrieval. IEEE Trans. Cybern. 45(4), 767–779 (2015)

    Article  Google Scholar 

  41. Yu, J., Yang, X., Fei, G., Tao, D.: Deep multimodal distance metric learning using click constraints for image ranking. IEEE Trans. Cybern. PP(99), 1–11 (2016)

    Google Scholar 

  42. Yu, J., Zhang, B., Kuang, Z., Lin, D., Fan, J.: iprivacy: image privacy protection by identifying sensitive objects via deep multi-task learning. IEEE Trans. Inf. Forensics Secur. 12(5), 1005–1016 (2017)

    Article  Google Scholar 

  43. Yuan, X.T., Liu, X., Yan, S.: Visual classification with multitask joint sparse representation. IEEE Trans. Image Process. 21(10), 4349–4360 (2012)

    Article  MathSciNet  MATH  Google Scholar 

  44. Zhang, T., Ghanem, B., Liu, S., Ahuja, N.: Robust visual tracking via structured multi-task sparse learning. Int. J. Comput. Vis. 101(2), 367–383 (2013)

    Article  MathSciNet  Google Scholar 

  45. Zheng, L., Huang, Y., Lu, H., Yang, Y.: Pose invariant embedding for deep person re-identification. CoRR abs/1701.07732 (2017). http://arxiv.org/abs/1701.07732

  46. Zheng, L., Yang, Y., Hauptmann, A.G.: Person re-identification: past, present and future (2016)

  47. Zheng, L., Zhang, H., Sun, S., Chandraker, M., Yang, Y., Tian, Q.: Person re-identification in the wild (2016)

  48. Zhong, W., Kwok, J.: Convex multitask learning with flexible task clusters. In: International Conference on Machine Learning, pp. 49–56 (2012)

  49. Zhou, J., Chen, J., Ye, J.: Clustered multi-task learning via alternating structure optimization. In: Advances in Neural Information Processing Systems, p. 702 (2011)

  50. Zhou, J., Chen, J., Ye, J.: Malsar: multi-task learning via structural regularization, vol. 21. Arizona State University, Tempe (2011)

    Google Scholar 

  51. Zhu, H., Xiao, F., Sun, L., Wang, R., Yang, P.: R-ttwd: robust device-free through-the-wall detection of moving human with wifi. IEEE J. Sel Areas Commun. PP(99), 1–1 (2017)

    Article  Google Scholar 

  52. Zitnick, C.L., Dollr, P.: Edge boxes: locating object proposals from edges. In: European Conference on Computer Vision, pp. 391–405 (2014)

Download references

Acknowledgements

This work is supported by the National Natural Science Foundation of China (61622205, 61472110), the Fujian Provincial Natural Science Foundation of China (2016J01327, 2016J01324), the Fujian Provincial High School Natural Science Foundation of China (JZ160472), and Foundation of Fujian Educational Committee (JAT160357, JAT160358).

Author information

Authors and Affiliations

  1. Xiamen University of Technology, Ligong Road #600, Houxi, Jimei, Xiamen, Fujian, China

    Liang Hu, Chaoqun Hong, Zhiqiang Zeng & Xiaodong Wang

Authors
  1. Liang Hu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Chaoqun Hong
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Zhiqiang Zeng
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Xiaodong Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Chaoqun Hong.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Hu, L., Hong, C., Zeng, Z. et al. Two-stream person re-identification with multi-task deep neural networks. Machine Vision and Applications 29, 947–954 (2018). https://doi.org/10.1007/s00138-018-0915-1

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00138-018-0915-1

Keywords

Search

Navigation