Skip to main content
SpringerLink
Log in

Multi-scale feature pyramid and multi-branch neural network for person re-identification

  • Original article
  • Published:
The Visual Computer Aims and scope Submit manuscript

Abstract

The key to person re-identification (Re-ID) is how to extract a representative and robust depth feature of the person, which requires the model to pay attention to both global contour information and local detailed features. To extract more representative features, an effective method is to build a multi-branch deep model by duplicating the backbone structure. This method usually severely increases the computational cost, and continuous convolution and pooling operations cause the loss of detailed information. This paper proposes a lightweight multi-scale feature pyramid structure, which extracts features from network layers of different scales and aggregates them to supplement spatial detail information. Meanwhile, this paper adopts a pair of complementary attention modules, which pay attention to the discriminative areas of person features by focusing on channel aggregation and position perception, respectively. In addition, this paper proposes a multi-level orthogonal regularization method to further enhance the diversity of features. The experimental results show that the mAP of this method on the Market1501 dataset reaches 91.6%. The proposed method outperforms state-of-the-art methods and along with lower complexity.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

References

  1. Zheng, L., Yang, Y., Hauptmann, A G.: Person re-identification: past, present and future[J]. (2016)

  2. Ye, M., Shen, J., Lin, G., et al.: Deep learning for person re-identification: a survey and outlook. IEEE Trans. Pattern Anal. Mach. Intell. 44(6), 2872–2893 (2021)

    Article  Google Scholar 

  3. Huang, H., Li, D., Zhang, Z., et al.: Adversarially occluded samples for person re-identification[C]. In: 2018 IEEE/CVF conference on computer vision and pattern recognition (CVPR). IEEE, (2018)

  4. Hou, R., Ma, B., Chang, H., et al.: VRSTC: occlusion-free video person re-identification[J].In: 2019 IEEE/cvf conference on computer vision and pattern recognition (CVPR), (2019)

  5. Zhao, H., Tian, M., Sun, S., et al.: Spindle net: person re-identification with human body region guided feature decomposition and fusion[C]. In: 2017 IEEE conference on computer vision and pattern recognition (CVPR). IEEE, (2017)

  6. Song, C., Yan, H., Ouyang, W., et al.: Mask-guided contrastive attention model for person re-identification[C]. In: 2018 IEEE/CVF conference on computer vision and pattern recognition (CVPR). IEEE, (2018)

  7. Gou, M., Fei, X., Camps, O., et al.: Person re-identification using kernel-based metric learning methods[C]. In: computer vision–ECCV 2014. (Springer, Cham, 2014)

  8. Rui, Z., Ouyang, W., Wang, X.: Person re-identification by salience matching[C]. In: proceedings of the 2013 IEEE international conference on computer vision. IEEE, (2013)

  9. Guillaumin, M., Verbeek, J., Schmid, C.: Is that you? Metric learning approaches for face identification[C]. In: IEEE international conference on computer vision. IEEE, (2009)

  10. Chen, J., Zhang, Z., Wang, Y.: Relevance metric learning for person re-identification by exploiting global similarities[C]. In: 2014 22nd international conference on pattern recognition. IEEE, (2014)

  11. Krizhevsky, A., Sutskever, I., Hinton, G.: ImageNet classification with deep convolutional neural networks. Adv. Neural Inf. Processing Syst. 25, 2 (2012)

    Google Scholar 

  12. Szegedy, C., Liu, W., Jia, Y., et al.: Going deeper with convolutions[J]. In: IEEE computer society, (2014)

  13. Sun, Y., Zheng, L., Yang, Y., et al.: Beyond part models: person retrieval with refined part pooling (and a strong convolutional baseline). Springer, Cham (2017)

    Google Scholar 

  14. Hao, L.: Bags of tricks and a strong baseline for deep person re-identification[C]. In: 2019 IEEE/CVF conference on computer vision and pattern recognition workshops (CVPRW). IEEE, 2019

  15. Chi, S., Li, J., Zhang. S,, et al.: Pose-driven deep convolutional model for person re-identification[C]. In: 2017 IEEE international conference on computer vision (ICCV). IEEE, (2017)

  16. Zhang, Z., Lan, C., Zeng, W., et al.: Relation-aware global attention for person re-identification[C]. In: 2020 IEEE/CVF conference on computer vision and pattern recognition (CVPR). IEEE, (2020)

  17. Chen, T.et al.: ABD-Net: attentive but diverse person re-identification. In: 2019 IEEE/CVF International Conference on Computer Vision (ICCV), pp. 8350–8360, https://doi.org/10.1109/ICCV.2019.00844 (2019)

  18. Wang, G., Yuan., Y, Chen, X., et al.: Learning discriminative features with multiple granularities for person re-identification[C]. In: 2018 ACM multimedia conference. ACM, (2018)

  19. Yang, W., Huang, H., Zhang, Z., et al.: Towards rich feature discovery with class activation maps augmentation for person re-identification[C]. In: IEEE conference on computer vision and pattern recognition 2019. IEEE, (2019)

  20. Zheng, F., Deng, C., Sun, X., et al.: Pyramidal person re-identification via multi-loss dynamic training[C]. In: 2019 IEEE/CVF conference on computer vision and pattern recognition (CVPR). IEEE, (2019)

  21. Wang, P., Zhao, Z., Fei, S., et al.: HOReID: deep high-order mapping enhances pose alignment for person re-identification. IEEE Trans. Image Processing 30, 2908–2922 (2021)

    Article  MathSciNet  Google Scholar 

  22. He, K., Zhang, X., Ren, S., et al.: Deep residual learning for image recognition. In: 2016 IEEE conference on computer vision and pattern recognition (CVPR), pp. 770-778, https://doi.org/10.1109/CVPR.2016.90 (2016)

  23. Cai, H., Wang, Z., Cheng, J.: Multi-scale body-part mask guided attention for person re-identification[C]. In: 2019 IEEE/CVF conference on computer vision and pattern recognition workshops (CVPRW). IEEE, (2020)

  24. Cheng, W.A., Ls, B., Gw, B., et al.: Multi-scale multi-patch person re-identification with exclusivity regularized softmax. Neurocomputing 382, 64–70 (2020)

    Article  Google Scholar 

  25. Liu, X., Tan, H., Tong, X., et al.: Feature preserving GAN and multi-scale feature enhancement for domain adaption person re-identification. Neurocomputing 364, 108–118 (2019)

    Article  Google Scholar 

  26. Wu, D., Wang, C., Wu, Y., et al.: Attention deep model with multi-scale deep supervision for person re-identification. IEEE Trans. Emerg. Topics Comput. Intell. 5(1), 70–78 (2021)

    Article  Google Scholar 

  27. Zhou, K., Yang, Y., Cavallaro, A., et al.: Omni-scale feature learning for person re-identification[C]. In: 2019 IEEE/CVF international conference on computer vision (ICCV). IEEE, 2020

  28. Chen, Z., Lv, X., Sun, T., et al.: FLAG: feature learning with additional guidance for person search. Vis. Comput. 37(4), 685–693 (2021)

    Article  Google Scholar 

  29. Zheng, M., Karanam, S., Wu, Z., et al.: Re-identification with consistent attentive siamese networks[C]. In: 2019 IEEE/CVF conference on computer vision and pattern recognition (CVPR). IEEE, (2019)

  30. Liu, H., Feng, J., Qi, M., et al.: End-to-End comparative attention networks for person re-identification. IEEE Trans. Image Processing Publ. IEEE Signal Processing Soc. 26(99), 3492–3506 (2017)

    Article  MathSciNet  MATH  Google Scholar 

  31. Li, W., Zhu, X., Gong, S.: Harmonious attention network for person re-identification[C].In: 2018 IEEE/CVF conference on computer vision and pattern recognition. IEEE, (2018)

  32. Xia., BN. Gong, Y., et al.: Second-order non-local Attention Networks for Person Re-identification[J]. In: 2019 IEEE/CVF international conference on computer vision (ICCV), (2019)

  33. Chen, B., Deng, W., Hu, J.: Mixed high-order attention network for person re-identification[J]. In: 2019 IEEE/CVF international conference on computer vision (ICCV), (2019)

  34. Zhang, L., Wu, X., Zhang, S., et al.: Branch-Cooperative OSNet for Person Re-Identification[J]. (2020)

  35. Guo, J., Yuan, Y., Huang, L., et al.: Beyond human parts: dual part-aligned representations for person re-identification[J]. In: 2019 IEEE/CVF international conference on computer vision (ICCV), (2019)

  36. Xie, J., Ge, Y., Zhang, J., et al.: Low-resolution assisted three-stream network for person re-identification. Vis. Comput. 38(7), 2515–2525 (2022)

    Article  Google Scholar 

  37. Quan, R., Dong, X., Wu, Y., et al.: Auto-ReID: searching for a part-aware ConvNet for person re-identification[C]. In: 2019 IEEE/CVF International Conference on Computer Vision (ICCV). IEEE, 2019.

  38. Van Der Walt, S., Colbert, S.C., Varoquaux, G.: The numpy array: a structure for efficient numerical computation. Comput. Sci. Eng. 13(2), 22–30 (2011)

    Article  Google Scholar 

  39. Ozay, M., Okatani, T.: Optimization on Submanifolds of Convolution Kernels in CNNs[J]. (2016)

  40. Xuan Z, Hao L, Xing F, et al. AlignedReID: Surpassing Human-Level Performance in Person Re-Identification[J]. 2017.

  41. Wen, Y., Zhang, K., Li, Z., et al.: A discriminative feature learning approach for deep face recognition[J]. (2016)

  42. Zheng, L., Shen, L., Lu, T., et al.: Scalable person re-identification: a benchmark[C]. In: 2015 IEEE international conference on computer vision (ICCV). IEEE, (2015)

  43. Ristani, E., Solera, F., Zou, R., et al. Performance measures and a data set for multi-target, multi-camera Tracking[J]. (Springer, Cham, 2016)

  44. Wei, L., Rui, Z., Tong, X., et al.: DeepReID: deep filter pairing neural network for person re-identification[C]. In: computer vision & pattern recognition. IEEE, (2014)

  45. Wei, L., Zhang, S., Wen, G., et al.: Person transfer GAN to bridge domain gap for person re-identification[J]. IEEE, (2018)

  46. Cheng, W., Qian, Z., Chang, H., et al.: Mancs: a multi-task attentional network with curriculum sampling for person re-identification: 15th European Conference, Munich, Germany, September 8–14, 2018, proceedings, Part IV[C]. In: european conference on computer vision. (Springer, Cham, 2018)

  47. Dai, Z., Chen, M., Gu, X., et al.: Batch dropblock network for person re-identification and beyond[C].In: 2019 IEEE/CVF international conference on computer vision (ICCV). IEEE, (2019)

  48. Zhang, S., Zhang, L., Wang, W., et al.: AsNet: asymmetrical network for learning rich features in person re-identification. IEEE Signal Processing Lett. 27, 850–854 (2020)

    Article  Google Scholar 

  49. Ni, X., Fang, L., Huttunen, H.: Adaptive L2 Regularization in Person Re-Identification[C]. In: 2020 25th international conference on pattern recognition (ICPR). (2021)

  50. Li, H., Wu, G., Zheng, W S.: Combined depth space based architecture search for person re-identification[J]. (2021)

  51. Li,Y., He, J., Zhang, T., et al. Diverse part discovery: occluded person re-identification with part-aware transformer[J]. (2021)

  52. Wei, L., Zhang, S., Yao, H., et al.: GLAD: global-local-alignment descriptor for scalable person re-identification[J]. IEEE Trans. Multimed 21(4), 986–999 (2019)

    Article  Google Scholar 

  53. Huang, H., Yang, W., Lin, J., et al.: Improve Person Re-Identification With Part Awareness Learning. IEEE Trans. Image Processing 29, 7468–7481 (2020)

    Article  MATH  Google Scholar 

  54. Zheng, Z., Yang, X., Yu, Z., et al.: Joint discriminative and generative learning for person re-identification[C]. In: 2019 IEEE/CVF conference on computer vision and pattern recognition (CVPR). IEEE, (2020)

Download references

Acknowledgements

This work was supported by National key research and development plan project (2016YFB1200602-37).

Funding

National key research and development plan project, 2016YFB1200602-37, Minglian Wang.

Author information

Authors and Affiliations

  1. Faculty of Information Technology, Beijing University of Technology, Beijing, China

    Pengfei Wang, Minglian Wang & Dongzhi He

Authors
  1. Pengfei Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Minglian Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Dongzhi He
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Dongzhi He.

Ethics declarations

Conflict of interest

The authors declare no competing interests.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wang, P., Wang, M. & He, D. Multi-scale feature pyramid and multi-branch neural network for person re-identification. Vis Comput 39, 5185–5197 (2023). https://doi.org/10.1007/s00371-022-02653-5

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00371-022-02653-5

Keywords

Search

Navigation