Skip to main content

Advertisement

Springer Nature Link
Log in

Exemplar-Free Lifelong Person Re-identification via Prompt-Guided Adaptive Knowledge Consolidation

  • Published:
International Journal of Computer Vision Aims and scope Submit manuscript

Abstract

Lifelong person re-identification (LReID) refers to matching people across different cameras given continuous data streams. The challenge of catastrophic forgetting of old knowledge and the effective acquisition of new knowledge form a significant dilemma for LReID. Most current LReID methods propose to retain abundant exemplars from historical data, which are further rehearsed to fully fine-tune the whole model. However, such a learning paradigm will inevitably hinder data privacy and result in substantial computation costs. In this paper, we propose a paradigm for exemplar-free LReID through model re-parameterization. Without retaining any exemplars, our designed method adopts a novel Prompt-guided Adaptive Exponential Moving Average (PAEMA) strategy to achieve dynamic knowledge consolidation. Our key idea is to leverage visual prompting as the guidance for model re-parameterization to benefit knowledge preservation. Conventional Exponential Moving Average (EMA) methods rely on fixed or time-varied constants as weighting parameters, the unpredictable correlation between new and old data streams may lead to varying levels of model parameter drifting during LReID learning. Hence, we argue that a proper weighting parameter should be conditioned on the variation of new and old models to provide an adaptive knowledge consolidation for LReID. To do so, an adaptive mechanism is proposed to utilize the visual prompt as a surrogate for model variation estimation. Consequently, without using any exemplars, the forgetting issue in LReID is greatly alleviated. Experiments on various LReID benchmarks have verified the superiority of our method against the state-of-the-art lifelong learning and LReID approaches. Code is available at https://github.com/zhoujiahuan1991/IJCV2024-PAEMA/.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Algorithm 1
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

Data Availability

No datasets are generated and the used datasets that support the findings are Market-1501 http://www.liangzheng.org/Project/project_reid.html, DukeMTMC http://vision.cs.duke.edu/DukeMTMC/, CUHK-SYSU https://drive.google.com/file/d/1XmiNVrfK2ZmI0ZZ2HHT80HHbDrnE4l3W/view?usp=sharing, MSMT\(\_\)v1, MSMT\(\_\)v2 http://www.pkuvmc.com/publications/msmt17.html, CUHK01, CUHK02, CUHK03 http://www.ee.cuhk.edu.hk/~xgwang/CUHK_identification.html, GRID http://personal.ie.cuhk.edu.hk/~ccloy/downloads_qmul_underground_reid.html, SenseReID https://drive.google.com/file/d/0B56OfSrVI8hubVJLTzkwV2VaOWM/view?resourcekey=0-PKtdd5m_Jatmi2n9Kb_gFQ, VIPeR http://users.soe.ucsc.edu/~manduchi/VIPeR.v1.0.zip, PRID https://www.tugraz.at/institute/icg/research/team-bischof/lrs/downloads/PRID11/, i-LIDS http://www.eecs.qmul.ac.uk/~jason/data/i-LIDS_Pedestrian.tgz. Code is available at https://github.com/zhoujiahuan1991/IJCV2024-PAEMA/.

References

  • Ahmed, E., Jones, M., & Marks, T.K. (2015). An improved deep learning architecture for person re-identification. In: CVPR, IEEE, pp. 3908–3916.

  • Amari, S.-i. (1993). Backpropagation and stochastic gradient descent method. Neurocomputing 5(4-5), 185–196

  • Cai, Z., Ravichandran, A., Maji, S., Fowlkes, C., Tu, Z., & Soatto, S. (2021). Exponential moving average normalization for self-supervised and semi-supervised learning. In: CVPR, IEEE, pp. 194–203.

  • Chaudhry, A., Dokania, P.K., Ajanthan, T., & Torr, P.H.S. (2018). Riemannian walk for incremental learning: Understanding forgetting and intransigence. In: Proceedings of the European Conference on Computer Vision (ECCV)

  • Chen, Y.-C., Zhu, X., Zheng, W.-S., & Lai, J.-H. (2017). Person re-identification by camera correlation aware feature augmentation. PAMI, 40(2), 392–408.

    Article  Google Scholar 

  • Cho, Y., Kim, W.J., Hong, S., & Yoon, S.-E. (2022). Part-based pseudo label refinement for unsupervised person re-identification. In: CVPR, pp. 7308–7318.

  • Cui, L., Wu, Y., Liu, J., Yang, S., & Zhang, Y. (2021). Template-based named entity recognition using bart. arXiv:2106.01760

  • Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009). Imagenet: A large-scale hierarchical image database. In: CVPR, pp. 248–255. IEEE

  • Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., et al. (2020). An image is worth 16x16 words: Transformers for image recognition at scale. arXiv:2010.11929

  • Douillard, A., Ramé, A., Couairon, G., & Cord, M. (2022). Dytox: Transformers for continual learning with dynamic token expansion. In: CVPR, IEEE, pp. 9285–9295.

  • Ge, W., Du, J., Wu, A., Xian, Y., Yan, K., Huang, F., & Zheng, W.-S. (2022). Lifelong person re-identification by pseudo task knowledge preservation. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 36, pp. 688–696.

  • Glorot, X., Bengio, Y. (2010). Understanding the difficulty of training deep feedforward neural networks. In: ICAIS, JMLR Workshop and Conference Proceedings, pp. 249–256.

  • Gray, D., & Tao, H. (2008). Viewpoint invariant pedestrian recognition with an ensemble of localized features. In: Computer Vision–ECCV 2008: 10th European Conference on Computer Vision, Marseille, France, October 12-18, 2008, Proceedings, Part I 10, pp. 262–275. Springer

  • He, S., Luo, H., Wang, P., Wang, F., Li, H., & Jiang, W. (2021). Transreid: Transformer-based object re-identification. In: ICCV, IEEE, pp. 14993–15002.

  • He, K., Zhang, X., Ren, S., & Sun, J. (2016). Deep residual learning for image recognition. In: CVPR, IEEE, pp. 770–778.

  • Hirzer, M., Beleznai, C., Roth, P.M., & Bischof, H. (2011). Person re-identification by descriptive and discriminative classification. In: Image Analysis: 17th Scandinavian Conference, SCIA 2011, Ystad, Sweden, May 2011. Proceedings 17, pp. 91–102. Springer

  • Houlsby, N., Giurgiu, A., Jastrzebski, S., Morrone, B., De Laroussilhe, Q., Gesmundo, A., Attariyan, M., & Gelly, S. (2019). Parameter-efficient transfer learning for nlp. In: ICML, PMLR, pp. 2790–2799.

  • Hu, Z., Li, Y., Lyu, J., Gao, D., & Vasconcelos, N. (2023). Dense network expansion for class incremental learning. In: CVPR, pp. 11858–11867.

  • Huang, Z., Zhang, Z., Lan, C., Zeng, W., Chu, P., You, Q., Wang, J., Liu, Z., & Zha, Z.-j. (2022). Lifelong unsupervised domain adaptive person re-identification with coordinated anti-forgetting and adaptation. In: CVPR, IEEE, pp. 14288–14297.

  • Isobe, T., Li, D., Tian, L., Chen, W., Shan, Y., & Wang, S. (2021). Towards discriminative representation learning for unsupervised person re-identification. In: ICCV, IEEE, pp. 8506–8516.

  • Jia, M., Tang, L., Chen, B.-C., Cardie, C., Belongie, S., Hariharan, B., & Lim, S.-N. (2022). Visual prompt tuning. arXiv:2203.12119

  • Jin, X., Lan, C., Zeng, W., Chen, Z., & Zhang, L. (2020). Style normalization and restitution for generalizable person re-identification. In: CVPR, pp. 3143–3152.

  • Kalb, T., & Beyerer, J. (2023). Principles of forgetting in domain-incremental semantic segmentation in adverse weather conditions. In: CVPR, pp. 19508–19518.

  • Kingma, D.P., & Ba, J. (2014). Adam: A method for stochastic optimization. arXiv:1412.6980

  • Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A. A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al. (2017). Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences, 114(13), 3521–3526.

    Article  MathSciNet  Google Scholar 

  • Lester, B., Al-Rfou, R., & Constant, N. (2021). The power of scale for parameter-efficient prompt tuning. arXiv:2104.08691

  • Li, W., & Wang, X. (2013). Locally aligned feature transforms across views. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3594–3601

  • Li, W., Zhao, R., & Wang, X. (2012). Human reidentification with transferred metric learning. In: ACCV, Springer, pp. 31–44.

  • Li, W., Zhao, R., Xiao, T., & Wang, X. (2014). Deepreid: Deep filter pairing neural network for person re-identification. In: CVPR, IEEE, pp. 152–159.

  • Li, W., Zhu, X., & Gong, S. (2018). Harmonious attention network for person re-identification. In: CVPR, IEEE, pp. 2285–2294.

  • Liao, S., & Shao, L. (2022). Graph sampling based deep metric learning for generalizable person re-identification. In: CVPR, pp. 7359–7368.

  • Li, Z., & Hoiem, D. (2017). Learning without forgetting. PAMI, 40(12), 2935–2947.

    Article  Google Scholar 

  • Lin, G., Chu, H., & Lai, H. (2022). Towards better plasticity-stability trade-off in incremental learning: A simple linear connector. In: CVPR, pp. 89–98.

  • Lin, Y., Dong, X., Zheng, L., Yan, Y., & Yang, Y. (2019). A bottom-up clustering approach to unsupervised person re-identification. In: AAAI, vol. 33, pp. 8738–8745.

  • Liu, Y., Schiele, B., Vedaldi, A., & Rupprecht, C. (2023). Continual detection transformer for incremental object detection. In: CVPR, pp. 23799–23808.

  • Liu, J., Zha, Z.-J., Chen, D., Hong, R., & Wang, M. (2019). Adaptive transfer network for cross-domain person re-identification. In: CVPR, IEEE, pp. 7195–7204.

  • Liu, Y., Schiele, B., & Sun, Q. (2021). Rmm: Reinforced memory management for class-incremental learning. Advances in Neural Information Processing Systems, 34, 3478–3490.

    Google Scholar 

  • Loy, C. C., Xiang, T., & Gong, S. (2010). Time-delayed correlation analysis for multi-camera activity understanding. IJCV, 90(1), 106–129.

    Article  Google Scholar 

  • Luo, H., Gu, Y., Liao, X., Lai, S., & Jiang, W. (2019). Bag of tricks and a strong baseline for deep person re-identification. In: CVPRW, pp. 1487–1495. IEEE

  • Luo, Z., Liu, Y., Schiele, B., & Sun, Q. (2023). Class-incremental exemplar compression for class-incremental learning. In: CVPR, pp. 11371–11380.

  • Malepathirana, T., Senanayake, D., & Halgamuge, S. (2023). Napa-vq: Neighborhood-aware prototype augmentation with vector quantization for continual learning. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 11674–11684

  • Ni, H., Song, J., Luo, X., Zheng, F., Li, W., & Shen, H.T. (2022). Meta distribution alignment for generalizable person re-identification. In: CVPR, pp. 2487–2496.

  • Petroni, F., Rocktäschel, T., Lewis, P., Bakhtin, A., Wu, Y., Miller, A.H., & Riedel, S. (2019). Language models as knowledge bases? arXiv:1909.01066

  • Prabhu, A., Torr, P.H., & Dokania, P.K. (2020). Gdumb: A simple approach that questions our progress in continual learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part II 16, pp. 524–540. Springer.

  • Pu, N., Chen, W., Liu, Y., Bakker, E.M., & Lew, M.S. (2021). Lifelong person re-identification via adaptive knowledge accumulation. In: CVPR, IEEE, pp. 7897–7906.

  • Pu, N., Liu, Y., Chen, W., Bakker, E.M., & Lew, M.S. (2022). Meta reconciliation normalization for lifelong person re-identification. In: ACMM, pp. 541–549.

  • Rannen, A., Aljundi, R., Blaschko, M.B., & Tuytelaars, T. (2017). Encoder based lifelong learning. In: ICCV, pp. 1320–1328.

  • Rebuffi, S.-A., Kolesnikov, A., Sperl, G., & Lampert, C.H. (2017). icarl: Incremental classifier and representation learning. In: CVPR, IEEE, pp. 5533–5542.

  • Ristani, E., Solera, F., Zou, R., Cucchiara, R., & Tomasi, C. (2016). Performance measures and a data set for multi-target, multi-camera tracking. In: ECCV, Springer, pp. 17–35.

  • Sankaranarayanan, S., Jain, A., & Lim, S.N. (2017). Guided perturbations: Self-corrective behavior in convolutional neural networks. In: ICCV, IEEE, pp. 3582–3590.

  • Shi, W., & Ye, M. (2023). Prototype reminiscence and augmented asymmetric knowledge aggregation for non-exemplar class-incremental learning. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 1772–1781

  • Shmelkov, K., Schmid, C., & Alahari, K. (2017). Incremental learning of object detectors without catastrophic forgetting. In: ICCV, IEEE, pp. 3420–3429.

  • Smith, J.S., Karlinsky, L., Gutta, V., Cascante-Bonilla, P., Kim, D., Arbelle, A., Panda, R., Feris, R., & Kira, Z. (2023). Coda-prompt: Continual decomposed attention-based prompting for rehearsal-free continual learning. In: CVPR, pp. 11909–11919

  • Song, J., Yang, Y., Song, Y.-Z., Xiang, T., & Hospedales, T.M. (2019). Generalizable person re-identification by domain-invariant mapping network. In: CVPR, IEEE, pp. 719–728.

  • Sun, Z., & Mu, Y. (2022). Patch-based knowledge distillation for lifelong person re-identification.

  • Sun, Z., Mu, Y., & Hua, G. (2023). Regularizing second-order influences for continual learning. In: CVPR, pp. 20166–20175

  • Tung, F., & Mori, G. (2019). Similarity-preserving knowledge distillation. In: ICCV, IEEE, pp. 1365–1374.

  • Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., & Polosukhin, I. (2017). Attention is all you need. NIPS 30

  • Wang F-Y, Zhou D-W, Liu L, Ye H-J, Bian Y, Zhan D-C, Zhao P. (2022). Beef: Bi-compatible class-incremental learning via energy-based expansion and fusion. In: The Eleventh International Conference on Learning Representations.

  • Wang, D., & Zhang, S. (2020). Unsupervised person re-identification via multi-label classification. In: CVPR, IEEE, pp. 10978–10987.

  • Wang, Z., He, L., Tu, X., Zhao, J., Gao, X., Shen, S., & Feng, J. (2021). Robust video-based person re-identification by ierarchical mining. CSVT

  • Wang, W., Hu, Y., Chen, Q., & Zhang, Y. (2023). Task difficulty aware parameter allocation & regularization for lifelong learning. In: CVPR, pp. 7776–7785.

  • Wang, Y., Huang, Z., & Hong, X. (2022). S-prompts learning with pre-trained transformers: An occam’s razor for domain incremental learning. arXiv:2207.12819

  • Wang, T., Yamaguchi, K., & Ordonez, V. (2018). Feedback-prop: Convolutional neural network inference under partial evidence. In: CVPR, IEEE, pp. 898–907.

  • Wang, Z., Zhang, Z., Ebrahimi, S., Sun, R., Zhang, H., Lee, C.-Y., Ren, X., Su, G., Perot, V., & Dy, J., et al. (2022). Dualprompt: Complementary prompting for rehearsal-free continual learning. arXiv:2204.04799

  • Wang, Z., Zhang, Z., Lee, C.-Y., Zhang, H., Sun, R., Ren, X., Su, G., Perot, V., Dy, J., & Pfister, T. (2022). Learning to prompt for continual learning. In: CVPR, IEEE, pp. 139–149.

  • Wang, F.-Y., Zhou, D.-W., Ye, H.-J., & Zhan, D.-C. (2022). Foster: Feature boosting and compression for class-incremental learning. In: European conference on computer vision, pp. 398–414. Springer

  • Wei, L., Zhang, S., Gao, W., & Tian, Q. (2018). Person transfer gan to bridge domain gap for person re-identification. In: CVPR, IEEE, pp. 79–88.

  • Wu, G., & Gong, S. (2021). Generalising without forgetting for lifelong person re-identification. In: AAAI, vol. 35, pp. 2889–2897.

  • Xiao, T., Li, S., Wang, B., Lin, L., & Wang, X. (2016). End-to-end deep learning for person search. 2(2), 4 arXiv:1604.01850

  • Xu, M., Zhang, Z., Hu, H., Wang, J., Wang, L., Wei, F., Bai, X., & Liu, Z. (2021). End-to-end semi-supervised object detection with soft teacher. In: ICCV, IEEE, pp. 3040–3049.

  • Yu, C., Shi, Y., Liu, Z., Gao, S., & Wang, J. (2023). Lifelong person re-identification via knowledge refreshing and consolidation. In: AAAI, vol. 37, pp. 3295–3303.

  • Yu, H.-X., Zheng, W.-S., Wu, A., Guo, X., Gong, S., & Lai, J.-H. (2019). Unsupervised person re-identification by soft multilabel learning. In: CVPR, IEEE, pp. 2143–2152.

  • Zhang, L., Gao, G., & Zhang, H. (2022). Spatial-temporal federated learning for lifelong person re-identification on distributed edges. arXiv:2207.11759

  • Zhang, W., He, X., Yu, X., Lu, W., Zha, Z., & Tian, Q. (2019). A multi-scale spatial-temporal attention model for person re-identification in videos. TIP, 29, 3365-3373.

    Google Scholar 

  • Zhao, B., Tang, S., Chen, D., Bilen, H., & Zhao, R. (2021). Continual representation learning for biometric identification. In: WACV, IEEE, pp. 1197–1207.

  • Zhao, H., Tian, M., Sun, S., Shao, J., Yan, J., Yi, S., Wang, X., & Tang, X. (2017). Spindle net: Person re-identification with human body region guided feature decomposition and fusion. In: CVPR, IEEE, pp. 907–915.

  • Zheng, W.-S., Gong, S., & Xiang, T. (2009). Associating groups of people.https://doi.org/10.5244/C.23.23

  • Zheng, K., Lan, C., Zeng, W., Zhang, Z., & Zha, Z.-J. (2021). Exploiting sample uncertainty for domain adaptive person re-identification. In: AAAI, vol. 35, pp. 3538–3546.

  • Zheng, L., Shen, L., Tian, L., Wang, S., Wang, J., & Tian, Q. (2015). Scalable person re-identification: A benchmark. In: ICCV, IEEE, pp. 1116–1124.

  • Zhou, D.-W., Wang, Q.-W., Ye, H.-J., & Zhan, D.-C. (2022). A model or 603 exemplars: Towards memory-efficient class-incremental learning. arXiv preprint arXiv:2205.13218

  • Zhu, H., Ke, W., Li, D., Liu, J., Tian, L., & Shan, Y. (2022). Dual cross-attention learning for fine-grained visual categorization and object re-identification. In: CVPR, pp. 4692–4702.

  • Zhuang, Z., Wei, L., Xie, L., Zhang, T., Zhang, H., Wu, H., Ai, H., & Tian, Q. (2020). Rethinking the distribution gap of person re-identification with camera-based batch normalization. In: ECCV, Springer, pp. 140–157.

Download references

Acknowledgements

This work was supported by the National Natural Science Foundation of China (62376011, 61925201, 62132001).

Author information

Authors and Affiliations

  1. Wangxuan Institute of Computer Technology, Peking University, Beijing, 100871, China

    Qiwei Li, Kunlun Xu, Yuxin Peng & Jiahuan Zhou

  2. National Key Laboratory for Multimedia Information Processing, Peking University, Beijing, 100871, China

    Qiwei Li, Kunlun Xu, Yuxin Peng & Jiahuan Zhou

Authors
  1. Qiwei Li
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Kunlun Xu
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Yuxin Peng
    View author publications

    You can also search for this author inPubMed Google Scholar

  4. Jiahuan Zhou
    View author publications

    You can also search for this author inPubMed Google Scholar

Corresponding author

Correspondence to Jiahuan Zhou.

Additional information

Communicated by Jingdong Wang.

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Li, Q., Xu, K., Peng, Y. et al. Exemplar-Free Lifelong Person Re-identification via Prompt-Guided Adaptive Knowledge Consolidation. Int J Comput Vis 132, 4850–4865 (2024). https://doi.org/10.1007/s11263-024-02110-x

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11263-024-02110-x

Keywords