Skip to main content
SpringerLink
Log in

Partial person re-identification using a pose-guided alignment network with mask learning

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

Partial person re-identification is a challenging task, in which only a partial observation of a person is available. There is severe misalignment when directly comparing a partial image with the holistic image, which leads to performance degradation with re-identification algorithms. In this paper, we propose a pose-guided alignment and mask learning network (PMN) to solve the problems of large parts missing and significant pedestrian misalignment. The proposed model includes a pose-guided spatial transformer (PST) module and a masked feature extractor. The PST module samples an affine transformed image from a holistic/partial image to align the pedestrian image with a standard pose. The masked feature extractor, which consists of a backbone network and a mask learning branch (MLB), is designed to learn the visibility of body parts to select effective features. The experimental results on two reported partial person benchmarks show that the proposed method achieves competitive performance compared to that of state-of-the-art methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13

Similar content being viewed by others

Code Availability

We will provide open source code on GitHub.

References

  1. Li W, Zhao R, Xiao T, Wang X (2014) Deepreid: Deep filter pairing neural network for person re-identification. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 152–159

  2. Ahmed E, Jones M, Marks TK (2015) An improved deep learning architecture for person re-identification. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR)

  3. Zhao H, Tian M, Sun S, Shao J, Yan J, Yi S, Wang X, Tang X (2017) Spindle net: Person re-identification with human body region guided feature decomposition and fusion. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 1077–1085

  4. Kalayeh MM, Basaran E, Gökmen M, Kamasak ME, Shah M (2018) Human semantic parsing for person re-identification. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 1062–1071

  5. Sun Y, Zheng L, Yang Y, Tian Q, Wang S (2018) Beyond part models: Person retrieval with refined part pooling (and a strong convolutional baseline). In: Proceedings of the European Conference on Computer Vision (ECCV), pp 480–496

  6. Jaderberg M, Simonyan K, Zisserman A et al (2015) Spatial transformer networks. In: Advances in neural information processing systems, pp 2017–2025

  7. Li D, Chen X, Zhang Z, Huang K (2017) Learning deep context-aware features over body and latent parts for person re-identification. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 384–393

  8. Zheng Z, Zheng L, Yang Y (2018) Pedestrian alignment network for large-scale person re-identification, IEEE Transactions on Circuits and Systems for Video Technology

  9. Wei L, Zhang S, Yao H, Gao W, Tian Q (2017) Glad: Global-local-alignment descriptor for pedestrian retrieval. In: Proceedings of the 25th ACM international conference on Multimedia. ACM, pp 420–428

  10. Su C, Li J, Zhang S, Xing J, Gao W, Tian Q (2017) Pose-driven deep convolutional model for person re-identification. In: Proceedings of the IEEE International Conference on Computer Vision, pp 3960–3969

  11. Zheng L, Huang Y, Lu H, Yang Y (2019) Pose invariant embedding for deep person re-identification, IEEE Transactions on Image Processing

  12. Saquib Sarfraz M, Schumann A, Eberle A, Stiefelhagen R (2018) A pose-sensitive embedding for person re-identification with expanded cross neighborhood re-ranking. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR)

  13. Wu J, Jiang J, Qi M, Liu H (2019) Independent metric learning with aligned multi-part features for video-based person re-identification. Multimedia Tools and Applications, pp 1–19

  14. Cheng D, Gong Y, Zhou S, Wang J, Zheng N (2016) Person re-identification by multi-channel parts-based cnn with improved triplet loss function. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 1335–1344

  15. Tian, Y, Li, Q, Wang, D, Wan, B, Robust joint learning network: improved deep representation learning for person re-identification, Multimedia Tools and Applications, pp 1–17

  16. Xiao J, Li H, Qu G, Fujita H, Cao Y, Zhu J, Huang C (2021) Hope: heatmap and offset for pose estimation. Journal of Ambient Intelligence and Humanized Computing, pp 1–13

  17. Song C, Huang Y, Ouyang W, Wang L (2018) Mask-guided contrastive attention model for person re-identification. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 1179–1188

  18. Zhao L, Li X, Zhuang Y, Wang J (2017) Deeply-learned part-aligned representations for person re-identification. In: Proceedings of the IEEE International Conference on Computer Vision, pp 3219–3228

  19. Si J, Zhang H, Li C-G, Kuen J, Kong X, Kot AC, Wang G (2018) Dual attention matching network for context-aware feature sequence based person re-identification. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 5363–5372

  20. Liu J, Ni B, Yan Y, Zhou P, Cheng S, Hu J (2018) Pose transferrable person re-identification. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 4099–4108

  21. Ma L, Sun Q, Georgoulis S, Van Gool L, Schiele B, Fritz M (2018) Disentangled person image generation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 99–108

  22. Qian X, Fu Y, Xiang T, Wang W, Qiu J, Wu Y, Jiang Y-G, Xue X (2018) Pose-normalized image generation for person re-identification. In: Proceedings of the European Conference on Computer Vision (ECCV), pp 650–667

  23. Ge Y, Li Z, Zhao H, Yin G, Yi S, Wang X (2018) Fd-gan: Pose-guided feature distilling gan for robust person re-identification. In: Advances in Neural Information Processing Systems, pp 1222–1233

  24. Zheng W-S, Li X, Xiang T, Liao S, Lai J, Gong S (2015) Partial person re-identification. In: The IEEE International Conference on Computer Vision (ICCV)

  25. He L, Liang J, Li H, Sun Z (2018) Deep spatial feature reconstruction for partial person re-identification: Alignment-free approach. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 7073–7082

  26. He L., Sun Z., Zhu Y, Wang Y (2018) Recognizing partial biometric patterns,” arXiv preprint arXiv:1810.07399

  27. Fan X, Luo H, Zhang X, He L, Zhang C, Jiang W (2018) Scpnet: Spatial-channel parallelism network for joint holistic and partial person re-identification. In: Asian Conference on Computer Vision. Springer, pp 19–34

  28. Sun Y, Xu Q, Li Y, Zhang C, Li Y, Wang S, Sun J (2019) Perceive where to focus: Learning visibility-aware part-level features for partial person re-identification. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 393–402

  29. Luo H, Jiang W, Fan X, Zhang C (2020) Stnreid: Deep convolutional networks with pairwise spatial transformer networks for partial person re-identification, IEEE Transactions on Multimedia

  30. Gao S, Wang J, Lu H, Liu Z (2020) Pose-guided visible part matching for occluded person reid. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp 11744–11752

  31. Fang H-S, Xie S, Tai Y-W, Lu C (2017) Rmpe: Regional multi-person pose estimation. In: Proceedings of the IEEE International Conference on Computer Vision, pp 2334–2343

  32. He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR)

  33. Xia BN, Gong Y, Zhang Y, Poellabauer C (2019) Second-order non-local attention networks for person re-identification. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp 3760–3769

  34. Zheng L, Shen L, Tian L, Wang S, Wang J, Tian Q (2015) Scalable person re-identification: a benchmark. In: Proceedings of the IEEE international conference on computer vision, pp 1116–1124

  35. Zheng W-S, Gong S, Xiang T (2011) Person re-identification by probabilistic relative distance comparison. In: CVPR 2011. IEEE, pp 649–656

  36. Luo H, Gu Y, Liao X, Lai S, Jiang W (2019) Bag of tricks and a strong baseline for deep person re-identification. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp 0–0

  37. Zhou K, Yang Y, Cavallaro A, Xiang T (2019) Omni-scale feature learning for person re-identification. In: Proceedings of the IEEE International Conference on Computer Vision, pp 3702–3712

  38. Chang X, Hospedales TM, Xiang T (2018) Multi-level factorisation net for person re-identification. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 2109–2118

  39. Li W, Zhu X, Gong S (2018) Harmonious attention network for person re-identification. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 2285–2294

  40. Suh Y, Wang J, Tang S, Mei T, Lee KM (2018) Part-aligned bilinear representations for person re-identification. In: Proceedings of the European Conference on Computer Vision (ECCV), pp 402–419

  41. Zhuo J, Lai J, Chen P (2019) A novel teacher-student learning framework for occluded person re-identification. arXiv preprint arXiv:1907.03253

  42. Miao J., Wu Y., Liu P., Ding Y., Yang Y. (2019) Pose-guided feature alignment for occluded person re-identification. In: Proceedings of the IEEE International Conference on Computer Vision, pp 542–551

Download references

Acknowledgements

The authors gratefully acknowledge funding from the National Natural Science Foundation of China under Grant Nos. 62071260, 62006131, and 61603202, the National Natural Science Foundation of Zhejiang Province under Grant Nos. LZ16F030001, LY17F030002, and LY20F030005 and the K. C. Wong Magna Fund of Ningbo University.

Author information

Authors and Affiliations

  1. Faculty of Electrical Engineering and Computer Science, NingBo University, Ningbo, 315000, China

    Qilu Qiu, Jieyu Zhao & Ye Zheng

Authors
  1. Qilu Qiu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jieyu Zhao
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ye Zheng
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jieyu Zhao.

Ethics declarations

Conflict of Interests

We declare no conflict of interest.

Additional information

Availability of Data and Material

We evaluate our method on the public Partial-REID dataset and Partial-iLIDS dataset and used the market-1501 dataset in the training process. The Partial-REID and Partial-iLIDS datasets are available at https://drive.google.com/file/d/1p7Jvo-RJhU_B6hf9eAhIEFNhvrzM5cdh/view.market-1501 dataset is available at https://drive.google.com/file/d/0B8-rUzbwVRk0c054eEozWG9COHM/view.

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Qiu, Q., Zhao, J. & Zheng, Y. Partial person re-identification using a pose-guided alignment network with mask learning. Appl Intell 52, 10885–10900 (2022). https://doi.org/10.1007/s10489-021-02928-9

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-021-02928-9

Keywords

Search

Navigation