Abstract
Learning distinguishing features from training datasets while filtering features of occlusions is critical to person retrieval scenarios. Most of the current person re-identification (Re-ID) methods based on classification or deep metric representation learning tend to overlook occlusion issues on the training set. Such representations from obstacles are easily over-fitted and misleading due to being considered as a part of the human body. To alleviate the occlusion problem, we propose a pose-guided feature region-based fusion network (PFRFN), to utilize pose landmarks as guidance to guide local learning for a good property of local feature, and the representation learning risk is evaluated on each part loss separately. Compared with only using global classification loss, concurrently considering local loss and the results of robust pose estimation enable the deep network to learn the representations of the body parts that prominently displayed in the image and gain the discriminative faculties on occluded scenes. Experimental results on multiple datasets, i.e., Market-1501, DukeMTMC, CUHK03, demonstrate the effectiveness of our method in a variety of scenarios.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Internaltional Conference on Computer Vision and Pattern Recogintion (CVPR), (2016)
Dai, Z., Chen, M., Gu, X., Zhu, S., Tan, P.: Batch dropblock network for person re-identification and beyond. In: International Conference on Computer Vision(ICCV), (2019)
Sun, Y., Zheng, L., Yang, Y., Tian, Q., Wang, S.: Beyond part models: Person retrieval with refined part pooling (and a strong convolutional baseline). In: European Conference on Computer Vision (ECCV), (2017)
Park, H., Ham, B.: Relation Network for Person Re-identification. In: the Association for the Advance of Artificial Intelligence (AAAI), (2020)
Zheng, Wei Shi., Xiang, Li., Tao, Xiang., Liao, Shengcai., Lai, Jianhuang., Gong, Shaogang.: Partial person re-identification. In: International Conference on Computer Vision(ICCV), (2015)
He, L., Liang, J., Li, H., Sun, Z.: Deep spatial feature reconstruction for partial person re-identification: Alignment-free approach. In: Internaltional Conference on Computer Vision and Pattern Recogintion (CVPR), (2018)
He, L., Sun, Z., Zhu, Y., Wang, Y.: Recognizing partial biometric patterns. CoRR, arXiv:1810.07399, (2018)
Zhang, S., Yang, J., Schiele, B.: Occluded pedestrian detection through guided attention in cnns. In: International Conference on Computer Vision and Pattern Recogintion (CVPR), (2018)
Miao, J., Wu, Y., Liu, P., Ding, Y., Yang, Y.: Pose-guided feature alignment for occluded person re-identification. In: International Conference on Computer Vision(ICCV), (2019)
Lin, Y., Zheng, L., Zhedong Zheng, Y.W., Zhilan, H., Yan, C., Yang, Y.: Improving person re-identification by attribute and identity learning. Pattern Recognit 95, 151–161 (2019)
Luo, Y., Zheng, L., Guan, T., Yu, J., Yang, Y.: Taking a closer look at domain shift: Category-level adversaries for semantics consistent domain adaptation. In: International Conference on Computer Vision and Pattern Recogintion (CVPR), (2019)
Qian, X., Fu, Y., Jiang, Y.-G., Xiang, T., Xue, X.: Multi-scale deep learning architectures for person re-identification. In: International Conference on Computer Vision (ICCV), (2017)
Tang, S., Andriluka, M., Andres, B., Schiele, B.: Multiple people tracking by lifted multicut and person re-identification. In: International Conference on Computer Vision and Pattern Recogintion (CVPR), (2017)
Varior, R.R., Bing, S., Lu, J., Dong, X., Gang, W.: A siamese long short-term memory architecture for human re-identification. In: European Conference on Computer Vision (ECCV), (2016)
Li, W., Zhu, X., Gong, S.: Person re-identification by deep joint learning of multi-loss classification. In: International Joint Conference on Artificial Intelligence (IJCAI), (2017)
Hadsell, R., Chopra, S., LeCun, Y.: Dimensionality reduction by learning an invariant mapping. In: International Conference on Computer Vision and Pattern Recogintion (CVPR), (2006)
Schroff, F., Kalenichenko, D., Philbin, J.: Facenet: A unified embedding for face recognition and clustering. In: International Conference on Computer Vision and Pattern Recogintion (CVPR), (2015)
Zhai, Y., Guo, X., Lu, Y., Li, H.: In defense of the classification loss for person re-identification. In: International Conference on Computer Vision and Pattern Recogintion (CVPR), (2019)
Chen, W., Chen, X., Zhang, J., Huang, K.: Beyond triplet loss: a deep quadruplet network for person re-identification. In: International Conference on Computer Vision and Pattern Recogintion (CVPR), (2017)
Hermans, A., Beyer, L., Leibe, B.: In defense of the triplet loss for person re-identification. arXiv:1703.07737 (2017)
Dong, Y., Zhen, L., Liao, S., Li, S.Z.: Deep metric learning for person re-identification. In: International Conference on Pattern Recognition (ICPR), (2014)
Hu, Z., Wu, H., Liao, S., Hu, H., Liu, S., Li, B.: Person re-identification with hybrid loss and hard triplets mining. In: International Conference on Multimedia Big Data (BigMM), (2018)
Xiao, Q., Luo, H., Zhang, C.: Margin sample mining loss: a deep learning based method for person re-identification. arXiv:1710.00478 (2017)
Wei, L., Rui, Z., Tong, X., Wang, X.: Deepreid: Deep filter pairing neural network for person re-identification. In: International Conference on Computer Vision and Pattern Recogintion (CVPR), (2014)
Fu, Y., Wei, Y., Zhou, Y., Shi, H., Huang, G., Wang, X., Yao, Z., Huang, T.: Horizontal pyramid matching for person re-identification. In: the Association for the Advance of Artificial Intelligence (AAAI), (2019)
Zhao, L., Li, X., Zhuang, Y., Wang, J.: Deeply-learned part-aligned representations for person re-identification. In: International Conference on Computer Vision (ICCV), (2017)
Suh, Y., Wang, J., Tang, S., Mei, T., Mu Lee, K.: Part-aligned bilinear representations for person re-identification. In: European Conference on Computer Vision (ECCV), (2018)
Zheng, Z., Zheng, L., Yang, Y.: Pedestrian alignment network for large-scale person re-identification. IEEE Trans. Circ. Syst. Video Technol. 29(10), 3037–3045 (2019)
Liu, X., Zhao, H., Tian, M., Sheng, L., Shao, J., Yi, S., Yan, J., Wang, X.: Hydraplus-net: Attentive deep features for pedestrian analysis. In: International Conference on Computer Vision (ICCV), (2017)
Li, W., Zhu, X., Gong, S.: Harmonious attention network for person re-identification. In: International Conference on Computer Vision and Pattern Recogintion (CVPR), (2018)
Chen, Y., Zhu, X., Gong, S.: Person re-identification by deep learning multi-scale representations. In: International Conference on Computer Vision (ICCV), (2017)
Su, C., Li, J., Zhang, S., Xing, J., Gao, W., Tian, Q.: Pose-driven deep convolutional model for person re-identification. In: International Conference on Computer Vision (ICCV), (2017)
Kalayeh, M., Basaran, E., Gökmen, M., Kamasak, M., Shah, M.: Human semantic parsing for person re-identification. In: International Conference on Computer Vision and Pattern Recogintion (CVPR), (2018)
Newell, A., Yang, K., Deng, J.: Stacked hourglass networks for human pose estimation. In: European Conference on Computer Vision (ECCV), (2016)
Zhao, H., Tian, M., Sun, S., Shao, J., Yan, J., Yi, S., Wang, X., Tang, X.: Spindle net: Person re-identification with human body region guided feature decomposition and fusion. In: International Conference on Computer Vision and Pattern Recogintion (CVPR), (2017)
Liu, J., Ni, B., Yan, Y., Peng, Z., Hu, J.: Pose transferrable person re-identification. In: International Conference on Computer Vision and Pattern Recogintion (CVPR), (2018)
Zhuo, J., Chen, Z., Lai, J.-H., Wang, G.: Occluded person re-identification. In: International Conference on Multimedia and Expo (ICME), (2018)
Huang, H., Li, D., Zhang, Z., Chen, X., Huang, K.: Adversarially occluded samples for person re-identification. In: International Conference on Computer Vision and Pattern Recogintion (CVPR), (2018)
Cao, Z., Martinez, G. Hidalgo., Simon, T., Wei, S., Sheikh, Y. A.: Openpose: Realtime multi-person 2d pose estimation using part affinity fields. IEEE Transactions on Pattern Analysis and Machine Intelligence, (2019)
Ma, L., Jia, X., Sun, Q., Schiele, B., Tuytelaars, T., Van Gool, L.: Pose guided person image generation. In: International Conference on Neural Information Processing Systems(NIPS), (2017)
Ristani, E., Solera, F., Zou, R., Cucchiara, R., Tomasi, C.: Performance measures and a data set for multi-target, multi-camera tracking. In: European Conference on Computer Vision (ECCV), (2016)
Li, W., Zhao, R., Xiao, T., Wang, X.: Deepreid: Deep filter pairing neural network for person re-identification. In: International Conference on Computer Vision and Pattern Recogintion (CVPR), (2014)
Zheng, L., Shen, L., Tian, L., Wang, S., Wang, J., Tian, Q.: Scalable person re-identification: A benchmark. In: International Conference on Computer Vision (ICCV), (2015)
Sun, Y., Zheng, L., Deng, W., Wang, S.: Svdnet for pedestrian retrieval. In: International Conference on Computer Vision (ICCV), (2017)
Bai, X., Yang, M., Huang, T., Dou, Z., Rui, Yu., Yongchao, X.: Deep-person: Learning discriminative deep features for person re-identification. Pattern Recogn. 98, 31–41 (2020)
Luo, Hao., Jiang, Wei., Zhang, Xuan., Fan, Xing., Qian, Jingjing., Zhang, Chi.: Alignedreid++: Dynamically matching local information for person re-identification. Pattern Recognition, 94:53–61, 05 (2019)
Wang, G., Yuan, Y., Chen, X., Li, J., Zhou, X.: Learning discriminative features with multiple granularities for person re-identification. In: ACM International Conference on Multimedia (ACM), (2018)
Wang, G., Yang, S., Liu, H., Wang, Z., Yang, Y., Wang, S., Yu, G., Zhou, E., Sun, J.: High-order information matters: Learning relation and topology for occluded person re-identification. In: International Conference on Computer Vision and Pattern Recogintion (CVPR), (2020)
Acknowledgements
This work was supported by the National Natural Science Foundation of China (61472278), the Major Project of Tianjin (18ZXZNGX00150), the Natural Science Foundation of Tianjin (18JCYBJC84800), the Scientific Research Key Project of Tianjin Municipal Education Commission (2017ZD13), and the Scientific Research Project of Tianjin Municipal Education Commission (2017KJ255).
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Xie, G., Wen, X., Yuan, L. et al. Pose-guided feature region-based fusion network for occluded person re-identification. Multimedia Systems 29, 1771–1783 (2023). https://doi.org/10.1007/s00530-021-00752-2
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00530-021-00752-2