Abstract
Person re-identification (re-id) with images is very useful in video surveillance to find specific targets. However, it is challenging due to the complex variations of human poses, camera viewpoints, lighting, occlusion, resolution, background clutter and so on. The key to tackle this problem is how to represent the body and match these representations among frames. Current methods usually use the features of the whole bodies, and the performance may be reduced because of part invisibility. To solve this problem, we propose a two-stream strategy to use parts and bodies simultaneously. It utilizes a multi-task learning framework with deep neural networks (DNNs). Part detection and body recognition are performed as two tasks, and the features are extracted by two DNNs. The features are connected to multi-task learning to compute the mapping model from features to identifications. With this model, re-id can be achieved. Experimental results on a challenging task show the effectiveness of the proposed method.
Similar content being viewed by others
References
Argyriou, A., Evgeniou, T., Pontil, M.: Multi-task feature learning. In: Conference on Advances in Neural Information Processing Systems, pp. 41–48 (2007)
Bengio, Y., et al.: Learning deep architectures for ai. Foundations and trends \(\textregistered \). Mach. Learn. 2(1), 1–127 (2009)
Bromley, J., Guyon, I., Lecun, Y., Sackinger, E., Shah, R.: Signature verification using a "siamese" time delay neural network. In: International Conference on Neural Information Processing Systems, pp. 737–744 (1993)
Caruana, R.: Multitask learning. Mach. Learn. 28(1), 41–75 (1997)
Chen, J., Liu, J., Ye, J.: Learning incoherent sparse and low-rank patterns from multiple tasks. ACM Trans. Knowl. Discov. Data 5(4), 22 (2012)
Chen, X., Lin, Q., Kim, S., Carbonell, J.G., Xing, E.P.: Smoothing proximal gradient method for general structured sparse learning. In: Twenty-Seventh Conference on Uncertainty in Artificial Intelligence, pp. 105–114 (2011)
Dalal, N., Triggs, B.: Histograms of oriented gradients for human detection. In: IEEE Conference on Computer Vision and Pattern Recognition, vol. 1, pp. 886–893. IEEE (2005)
Evgeniou, T., Pontil, M.: Regularized multi-task learning. In: International Conference on Knowledge Discovery and Data Mining, pp. 109–117 (2004)
Gong, P., Ye, J., Zhang, C.: Robust multi-task feature learning. In: International Conference on Knowledge Discovery & Data Mining, pp. 895–903 (2012)
Hermans, A., Beyer, L., Leibe, B.: In defense of the triplet loss for person re-identification. arXiv preprint arXiv:1703.07737 (2017)
Jalali, A., Sanghavi, S., Ruan, C., et al.: A dirty model for multi-task learning. In: Lafferty, J.D., Williams, C.K.I., Shawe-Taylor, J., Zemel, R.S., Culotta, A. (eds.) Advances in Neural Information Processing Systems, pp. 964–972. Curran Associates, Inc. (2010)
Ji, S., Ye, J.: An accelerated gradient method for trace norm minimization. In: International Conference on Machine Learning, pp. 457–464 (2009)
Jia, Y., Shelhamer, E., Donahue, J., Karayev, S., Long, J., Girshick, R., Guadarrama, S., Darrell, T.: Caffe: convolutional architecture for fast feature embedding. arXiv preprint arXiv:1408.5093 (2014)
Kang, Z., Grauman, K., Sha, F.: Learning with whom to share in multi-task feature learning. In: International Conference on Machine Learning, pp. 521–528 (2011)
Karpagavalli, P., Ramprasad, A.V.: An adaptive hybrid gmm for multiple human detection in crowd scenario. Multimed. Tools Appl. 76, 1–21 (2016)
Li, W., Zhao, R., Xiao, T., Wang, X.: Deepreid: deep filter pairing neural network for person re-identification. In: Computer Vision and Pattern Recognition, pp. 152–159 (2014)
Li, X., Zhao, L., Wei, L., Yang, M.H., Wu, F., Zhuang, Y., Ling, H., Wang, J.: Deepsaliency: multi-task deep neural network model for salient object detection. IEEE Trans Image Process 25(8), 3919 (2016)
Lin, W., Shen, Y., Yan, J., Xu, M., Wu, J., Wang, J., Lu, K.: Learning correspondence structures for person re-identification. IEEE Trans Image Process 26(5), 2438–2453 (2017)
Liu, W., Yang, X., Tao, D., Cheng, J., Tang, Y.: Multiview dimension reduction via hessian multiset canonical correlations. Inf Fusion 41, 119–128 (2017)
Mar, N.J., Vazquez, D., Lopez, A.M., Amores, J., Kuncheva, L.I.: Occlusion handling via random subspace classifiers for human detection. IEEE Trans. Cybern. 44(3), 342–354 (2017)
Miseikis, J., Borges, P.V.K.: Joint human detection from static and mobile cameras. IEEE Trans. Intell. Transp. Syst. 16(2), 1018–1029 (2015)
Sang, J., Xu, C., Liu, J.: User-aware image tag refinement via ternary semantic analysis. IEEE Trans. Multimed. 14(3), 883–895 (2012)
Shao, L., Wu, D., Li, X.: Learning deep and wide: a spectral method for learning deep networks. IEEE Trans. Neural Netw. Learn. Syst. 25(12), 2303–2308 (2014)
Srivastava, N., Hinton, G.E., Krizhevsky, A., Sutskever, I., Salakhutdinov, R.: Dropout: a simple way to prevent neural networks from overfitting. J. Mach. Learn. Res. 15(1), 1929–1958 (2014)
Sun, C., Wang, D., Lu, H.: Person re-identification via distance metric learning with latent variables. IEEE Trans. Image Process. 26(1), 23–34 (2016)
Tibshirani, R.: Regression shrinkage and selection via the lasso: a retrospective. J. R. Stat. Soc. 73(3), 267–288 (2011)
Varior, R.R., Wang, G., Lu, J., Liu, T.: Learning invariant color features for person reidentification. IEEE Trans. Image Process. 25(7), 3395–3410 (2016)
Xiao, F., Liu, W., Li, Z., Chen, L., Wang, R.: Noise-tolerant wireless sensor networks localization via multi-norms regularized matrix completion. IEEE Trans. Veh. Technol. PP(99), 1–1 (2017)
Xiao, F., Wang, Z., Ye, N., Wang, R., Li, X.Y.: One more tag enables fine-grained rfid localization and tracking. IEEE ACM Trans. Netw. PP(99), 1–14 (2017)
Xiao, T., Li, H., Ouyang, W., Wang, X.: Learning deep feature representations with domain guided dropout for person re-identification. In: CVPR (2016)
Xiao, T., Li, S., Wang, B., Lin, L., Wang, X.: Joint detection and identification feature learning for person search. In: CVPR (2017)
Xu, C.: Exploiting social-mobile information for location visualization. ACM 8, 39 (2017)
Yan, Y., Ricci, E., Liu, G., Sebe, N.: Egocentric daily activity recognition via multitask clustering. IEEE Trans. Image Process. 24(10), 2984–2995 (2015)
Yan, Y., Ricci, E., Subramanian, R., Liu, G., Sebe, N.: Multitask linear discriminant analysis for view invariant action recognition. IEEE Trans. Image Process. 23(12), 5599–5611 (2014)
Yang, X., Liu, W., Tao, D., Cheng, J.: Canonical correlation analysis networks for two-view image recognition. Inf. Sci. 385(C), 338–352 (2017)
Yi, D., Lei, Z., Liao, S., et al.: Deep metric learning for person re-identification. In: ICPR ’14 Proceedings of the 2014 22nd International Conference on Pattern Recognition, pp. 34–39. IEEE Computer Society, Washington, DC, USA (2014)
Yogarajah, P., Chaurasia, P., Condell, J., Prasad, G.: Enhancing gait based person identification using joint sparsity model and -norm minimization. Inf. Sci. 308, 3–22 (2015)
Yu, J., Rui, Y., Chen, B.: Exploiting click constraints and multi-view features for image re-ranking. IEEE Trans. Multimed. 16(1), 159–168 (2013)
Yu, J., Rui, Y., Tao, D.: Click prediction for web image reranking using multimodal sparse coding. IEEE Trans. Image Process. 23(5), 2019–32 (2014)
Yu, J., Tao, D., Wang, M., Rui, Y.: Learning to rank using user clicks and visual features for image retrieval. IEEE Trans. Cybern. 45(4), 767–779 (2015)
Yu, J., Yang, X., Fei, G., Tao, D.: Deep multimodal distance metric learning using click constraints for image ranking. IEEE Trans. Cybern. PP(99), 1–11 (2016)
Yu, J., Zhang, B., Kuang, Z., Lin, D., Fan, J.: iprivacy: image privacy protection by identifying sensitive objects via deep multi-task learning. IEEE Trans. Inf. Forensics Secur. 12(5), 1005–1016 (2017)
Yuan, X.T., Liu, X., Yan, S.: Visual classification with multitask joint sparse representation. IEEE Trans. Image Process. 21(10), 4349–4360 (2012)
Zhang, T., Ghanem, B., Liu, S., Ahuja, N.: Robust visual tracking via structured multi-task sparse learning. Int. J. Comput. Vis. 101(2), 367–383 (2013)
Zheng, L., Huang, Y., Lu, H., Yang, Y.: Pose invariant embedding for deep person re-identification. CoRR abs/1701.07732 (2017). http://arxiv.org/abs/1701.07732
Zheng, L., Yang, Y., Hauptmann, A.G.: Person re-identification: past, present and future (2016)
Zheng, L., Zhang, H., Sun, S., Chandraker, M., Yang, Y., Tian, Q.: Person re-identification in the wild (2016)
Zhong, W., Kwok, J.: Convex multitask learning with flexible task clusters. In: International Conference on Machine Learning, pp. 49–56 (2012)
Zhou, J., Chen, J., Ye, J.: Clustered multi-task learning via alternating structure optimization. In: Advances in Neural Information Processing Systems, p. 702 (2011)
Zhou, J., Chen, J., Ye, J.: Malsar: multi-task learning via structural regularization, vol. 21. Arizona State University, Tempe (2011)
Zhu, H., Xiao, F., Sun, L., Wang, R., Yang, P.: R-ttwd: robust device-free through-the-wall detection of moving human with wifi. IEEE J. Sel Areas Commun. PP(99), 1–1 (2017)
Zitnick, C.L., Dollr, P.: Edge boxes: locating object proposals from edges. In: European Conference on Computer Vision, pp. 391–405 (2014)
Acknowledgements
This work is supported by the National Natural Science Foundation of China (61622205, 61472110), the Fujian Provincial Natural Science Foundation of China (2016J01327, 2016J01324), the Fujian Provincial High School Natural Science Foundation of China (JZ160472), and Foundation of Fujian Educational Committee (JAT160357, JAT160358).
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Hu, L., Hong, C., Zeng, Z. et al. Two-stream person re-identification with multi-task deep neural networks. Machine Vision and Applications 29, 947–954 (2018). https://doi.org/10.1007/s00138-018-0915-1
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00138-018-0915-1