Skip to main content
SpringerLink
Log in

Prompt-Based Transformer for Generalizable Person Re-identification with Image Masking

  • Conference paper
  • First Online:
Biometric Recognition (CCBR 2023)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 14463))

Included in the following conference series:

  • 397 Accesses

Abstract

Domain generalizable (DG) person re-identification (ReID) aims to perform well on the unseen target domains by training on multiple source domains with different distribution, which is a realistic but challenging problem. Existing DG person ReID methods have not well explored the domain-specific knowledge based on the Transformer. In this paper, we propose a Prompt-based transformer framework which embeds domain-specific knowledge into different domain prompts, which are optionally optimized by different source domains. Furthermore, we exploit a pretext task of masking and predicting for DG ReID to broaden the understanding of model about data by learning from the signals of the corresponding matching image, which enables interaction between image pairs and improves the ability of generalization. Extensive experiments demonstrate that our method achieves state-of-the-art performances on the popular benchmarks.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 59.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 79.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Zhong, Z., Zheng, L., Cao, D., Li, S.: Re-ranking person re-identification with k-reciprocal encoding. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1318–1327 (2017)

    Google Scholar 

  2. Sun, Y., Zheng, L., Yang, Y., Tian, Q., Wang, S.: Beyond part models: person retrieval with refined part pooling (and a strong convolutional baseline). In: Proceedings of the European Conference on Computer Vision, ECCV 2018, pp. 480–496 (2018)

    Google Scholar 

  3. Miao, J., Wu, Y., Liu, P., Ding, Y., Yang, Y.: Pose-guided feature alignment for occluded person re-identification. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 542–551 (2019)

    Google Scholar 

  4. Hou, R., Ma, B., Chang, H., Gu, X., Shan, S., Chen, X.: IAUnet: global context-aware feature learning for person reidentification. IEEE Trans. Neural Netw. Learn. Syst. 32(10), 4460–4474 (2020)

    Article  Google Scholar 

  5. Zhong, Z., Zheng, L., Luo, Z., Li, S., Yang, Y.: Invariance matters: exemplar memory for domain adaptive person re-identification. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 598–607 (2019)

    Google Scholar 

  6. Dai, Y., Liu, J., Sun, Y., Tong, Z., Zhang, C., Duan, L. Y.: IDM: an intermediate domain module for domain adaptive person re-ID. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 11864–11874 (2021)

    Google Scholar 

  7. Jin, X., Lan, C., Zeng, W., Chen, Z., Zhang, L.: Style normalization and restitution for generalizable person re-identification. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3143–3152 (2020)

    Google Scholar 

  8. Dai, Y., Li, X., Liu, J., Tong, Z., Duan, L. Y.: Generalizable person re-identification with relevance-aware mixture of experts. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 16145–16154 (2021)

    Google Scholar 

  9. Bai, Y., et al.: Person30K: a dual-meta generalization network for person re-identification. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 2123–2132 (2021)

    Google Scholar 

  10. Choi, S., Kim, T., Jeong, M., Park, H., Kim, C.: Meta batch-instance normalization for generalizable person re-identification. In: Proceedings of the IEEE/CVF conference on Computer Vision and Pattern Recognition, pp. 3425–3435 (2021)

    Google Scholar 

  11. Carlucci, F.M., D’Innocente, A., Bucci, S., Caputo, B., Tommasi, T.: Domain generalization by solving jigsaw puzzles. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 2229–2238 (2019)

    Google Scholar 

  12. Maniyar, U., Deshmukh, A. A., Dogan, U., Balasubramanian, V.N.: Zero shot domain generalization. arXiv preprint arXiv:2008.07443 (2020)

  13. Dosovitskiy, A., et al.: An image is worth 16\(\times \)16 words: transformers for image recognition at scale. arXiv preprint arXiv:2010.11929 (2020)

  14. Zhou, K., Yang, J., Loy, C.C., Liu, Z.: Learning to prompt for vision-language models. Int. J. Comput. Vision 130(9), 2337–2348 (2022)

    Article  Google Scholar 

  15. Wei, L., Zhang, S., Gao, W., Tian, Q.: Person transfer GAN to bridge domain gap for person re-identification. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 79–88 (2018)

    Google Scholar 

  16. Zheng, L., Shen, L., Tian, L., Wang, S., Wang, J., Tian, Q.: Scalable person re-identification: a benchmark. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 1116–1124 (2015)

    Google Scholar 

  17. Li, W., Wang, X.: Locally aligned feature transforms across views. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3594–3601 (2013)

    Google Scholar 

  18. Li, W., Zhao, R., Xiao, T., Wang, X.: DeepReid: deep filter pairing neural network for person re-identification. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 152–159 (2014)

    Google Scholar 

  19. Xiao, T., Li, S., Wang, B., Lin, L., Wang, X.: End-to-end deep learning for person search. arXiv preprint arXiv:1604.01850, 2(2), 4 (2016)

  20. Hirzer, M., Beleznai, C., Roth, P.M., Bischof, H.: Person re-identification by descriptive and discriminative classification. In: Heyden, A., Kahl, F. (eds.) SCIA 2011. LNCS, vol. 6688, pp. 91–102. Springer, Heidelberg (2011). https://doi.org/10.1007/978-3-642-21227-7_9

    Chapter  Google Scholar 

  21. Loy, C.C., Xiang, T., Gong, S.: Time-delayed correlation analysis for multi-camera activity understanding. Int. J. Comput. Vision 90(1), 106–129 (2010)

    Article  Google Scholar 

  22. Gray, D., Tao, H.: Viewpoint invariant pedestrian recognition with an ensemble of localized features. In: Forsyth, D., Torr, P., Zisserman, A. (eds.) ECCV 2008. LNCS, vol. 5302, pp. 262–275. Springer, Heidelberg (2008). https://doi.org/10.1007/978-3-540-88682-2_21

    Chapter  Google Scholar 

  23. Zheng, W.S., Gong, S., Xiang, T.: Associating groups of people. In: BMVC, pp. 1–11 (2009)

    Google Scholar 

  24. Liao, S., Shao, L.: Interpretable and generalizable person re-identification with query-adaptive convolution and temporal lifting. In: Vedaldi, A., Bischof, H., Brox, T., Frahm, J.-M. (eds.) ECCV 2020, Part XI. LNCS, vol. 12356, pp. 456–474. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-58621-8_27

    Chapter  Google Scholar 

  25. Zhuang, Z., et al.: Rethinking the distribution gap of person re-identification with camera-based batch normalization. In: Vedaldi, A., Bischof, H., Brox, T., Frahm, J.-M. (eds.) ECCV 2020, Part XII. LNCS, vol. 12357, pp. 140–157. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-58610-2_9

    Chapter  Google Scholar 

  26. Jin, X., Lan, C., Zeng, W., Chen, Z., Zhang, L.: Style normalization and restitution for generalizable person re-identification. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3143–3152 (2020)

    Google Scholar 

  27. Shu, Y., Cao, Z., Wang, C., Wang, J., Long, M.: Open domain generalization with domain-augmented meta-learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 9624–9633 (2021)

    Google Scholar 

  28. Zhao, Y., Zhong, Z., Yang, F., Luo, Z., Lin, Y., Li, S., Sebe, N.: Learning to generalize unseen domains via memory-based multi-source meta-learning for person re-identification. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 6277–6286) (2021)

    Google Scholar 

  29. Gidaris, S., Singh, P., Komodakis, N.: Unsupervised representation learning by predicting image rotations. arXiv preprint arXiv:1803.07728 (2018)

  30. Chen, H., Lagadec, B., Bremond, F.: Ice: Inter-instance contrastive encoding for unsupervised person re-identification. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 14960–14969 (2021)

    Google Scholar 

  31. He, K., Chen, X., Xie, S., Li, Y., Dollár, P., Girshick, R.: Masked autoencoders are scalable vision learners. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 16000–16009 (2022)

    Google Scholar 

Download references

Acknowledgments

This work is partially supported by National Natural Science Foundation of China (Grants no. 62176271), and Science and Technology Program of Guangzhou (Grant no. 202201011681).

Author information

Authors and Affiliations

  1. School of Computer Science and Engineering, Sun Yat-sen University, Guangzhou, China

    Jialu Liu & Meng Yang

Authors
  1. Jialu Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Meng Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Meng Yang .

Editor information

Editors and Affiliations

  1. Hefei University of Technology, Hefei, China

    Wei Jia

  2. South China University of Technology, Guangzhou, China

    Wenxiong Kang

  3. China University of Mining and Technology, Xuzhou, China

    Zaiyu Pan

  4. Shandong University, Jinan, China

    Xianye Ben

  5. China University of Mining and Technology, Xuzhou, China

    Zhengfu Bian

  6. Southern University of Science and Technology, Shenzhen, China

    Shiqi Yu

  7. Chinese Academy of Sciences, Beijing, China

    Zhaofeng He

  8. China University of Mining and Technology, Xuzhou, China

    Jun Wang

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Liu, J., Yang, M. (2023). Prompt-Based Transformer for Generalizable Person Re-identification with Image Masking. In: Jia, W., et al. Biometric Recognition. CCBR 2023. Lecture Notes in Computer Science, vol 14463. Springer, Singapore. https://doi.org/10.1007/978-981-99-8565-4_25

Download citation

  • DOI: https://doi.org/10.1007/978-981-99-8565-4_25

  • Published:

  • Publisher Name: Springer, Singapore

  • Print ISBN: 978-981-99-8564-7

  • Online ISBN: 978-981-99-8565-4

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation