Skip to main content
SpringerLink
Log in

Flexible few-shot class-incremental learning with prototype container

  • Original Article
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

In the few-shot class-incremental learning, new class samples are utilized to learn the characteristics of new classes, while old class exemplars are used to avoid old knowledge forgetting. The limited number of new class samples is more likely to cause overfitting during incremental training. Moreover, mass stored old exemplars mean large storage space consumption. To solve the above difficulties, in this paper we propose a novel flexible few-shot class-incremental framework to make the incremental process efficient and convenient. We enhance the expression ability of extracted features through multistage pre-training. Then, we set up a prototype container to store each class prototype to retain old knowledge. When new classes flow in, we calculate the new class prototypes and update the prototype container. Finally, we get the prediction result through similarity weighting. The entire framework only need to train the base class classifier and does not require further training during the incremental process. It avoids the overfitting of novel classes and saves time for further training. Besides, storing prototypes can save more storage space than original image data. Overall, the entire framework has the advantage of flexibility. We conduct extensive experiments on three standard few-shot class-incremental datasets and achieve state-of-the-art results. Especially, to verify the flexibility of the framework, we discuss the special federated few-shot class-incremental scenarios in addition. No further training and less storage consumption provide the possibility for applications in more complex scenarios.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

Data availability

The datasets generated during and/or analyzed during the current study are available in the [The CIFAR100 dataset] repository with [http://www.cs.toronto.edu/~kriz/cifar.html], [Caltech-UCSD Birds-200-2011 (CUB-200-2011)] repository with [https://www.vision.caltech.edu/datasets/cub_200_2011/] and [miniImageNet] repository with [https://www.kaggle.com/datasets/arjunashok33/miniimagenet].

References

  1. McCormack J, Lomas A (2021) Deep learning of individual aesthetics. Neural Comput Appl 33(1):3–17

    Article  Google Scholar 

  2. Chen T, Frankle J, Chang S, Liu S, Zhang Y, Carbin M, Wang Z (2021) The lottery tickets hypothesis for supervised and self-supervised pre-training in computer vision models. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 16306–16316

  3. Zhang F, Xu X, Nauata N, Furukawa Y (2021) Structured outdoor architecture reconstruction by exploration and classification. In: Proceedings of the IEEE/CVF international conference on computer vision, pp 12427–12435

  4. Joseph K, Khan S, Khan FS, Balasubramanian VN (2021) Towards open world object detection. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 5830–5840

  5. Ye J, Yang X, Kang S, He Y, Zhang W, Huang L, Jiang M, Zhang W, Shi Y, Xia M et al (2021) A robust MTMC tracking system for AI-city challenge 2021. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 4044–4053

  6. Wang Z, Fang Z, Li D, Yang H, Du W (2021) Semantic supplementary network with prior information for multi-label image classification. IEEE Trans Circuits Syst Video Technol 32(4):1848–1859

    Article  Google Scholar 

  7. Zhu Y, Wang Z, Zha H, Gao D (2017) Boundary-eliminated pseudoinverse linear discriminant for imbalanced problems. IEEE Trans Neural Netw Learn Syst 29(6):2581–2594

    Article  MathSciNet  Google Scholar 

  8. Izzuddin TA, Safri NM, Othman MA (2021) Mental imagery classification using one-dimensional convolutional neural network for target selection in single-channel BCI-controlled mobile robot. Neural Comput Appl 33(11):6233–6246

    Article  Google Scholar 

  9. Ruping S (2001) Incremental learning with support vector machines. In: Proceedings 2001 IEEE international conference on data mining, IEEE, pp 641–642

  10. Wu Y, Chen Y, Wang L, Ye Y, Liu Z, Guo Y, Fu Y (2019) Large scale incremental learning. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 374–382

  11. Castro FM, Marín-Jiménez MJ, Guil N, Schmid C, Alahari K (2018) End-to-end incremental learning. In: Proceedings of the European conference on computer vision, pp 233–248

  12. Polikar R, Upda L, Upda SS, Honavar V (2001) Learn++: an incremental learning algorithm for supervised neural networks. IEEE Trans Syst Man Cybern Part C Appl Rev 31(4):497–508

    Article  Google Scholar 

  13. Wang Z, Cao C, Zhu Y (2020) Entropy and confidence-based undersampling boosting random forests for imbalanced problems. IEEE Trans Neural Netw Learn Syst 31(12):5178–5191

    Article  Google Scholar 

  14. Michieli U, Zanuttigh P (2021) Knowledge distillation for incremental learning in semantic segmentation. Comput Vis Image Underst 205:103167

    Article  Google Scholar 

  15. Javed K, Shafait F (2018) Revisiting distillation and incremental classifier learning. In: Asian conference on computer vision, Springer, pp 3–17

  16. Chen L, Yu C, Chen L (2019) A new knowledge distillation for incremental object detection. In: 2019 International joint conference on neural networks (IJCNN), IEEE, pp 1–7

  17. Xiang Y, Miao Y, Chen J, Xuan Q (2020) Efficient incremental learning using dynamic correction vector. IEEE Access 8:23090–23099

    Article  Google Scholar 

  18. Valipour S, Perez C, Jagersand M (2017) Incremental learning for robot perception through HRI. In: 2017 IEEE/RSJ international conference on intelligent robots and systems, IEEE, pp 2772–2777

  19. Han S, Meng Z, Khan A-S, Tong Y (2016) Incremental boosting convolutional neural network for facial action unit recognition. In: Advances in neural information processing systems, vol 29, pp 109–117

  20. Perez E, Kiela D, Cho K (2021) True few-shot learning with language models. In: Advances in neural information processing systems, vol 34, pp 11054–11070

  21. Sun X, Wang B, Wang Z, Li H, Li H, Fu K (2021) Research progress on few-shot learning for remote sensing image interpretation. IEEE J Sel Top Appl Earth Obs Remote Sens 14:2387–2402

    Article  Google Scholar 

  22. Ma J, Fong SH, Luo Y, Bakkenist CJ, Shen JP, Mourragui S, Wessels LF, Hafner M, Sharan R, Peng J et al (2021) Few-shot learning creates predictive models of drug response that translate from high-throughput screens to individual patients. Nat Cancer 2(2):233–244

    Article  Google Scholar 

  23. Tao X, Hong X, Chang X, Dong S, Wei X, Gong Y (2020) Few-shot class-incremental learning. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 12183–12192

  24. Cheraghian A, Rahman S, Fang P, Roy SK, Petersson L, Harandi M (2021) Semantic-aware knowledge distillation for few-shot class-incremental learning. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 2534–2543

  25. Li Y, Yang J (2021) Meta-learning baselines and database for few-shot classification in agriculture. Comput Electron Agric 182:106055

    Article  Google Scholar 

  26. Boudiaf M, Kervadec H, Masud ZI, Piantanida P, Ben Ayed I, Dolz J (2021) Few-shot segmentation without meta-learning: a good transductive inference is all you need? In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 13979–13988

  27. Chen Y, Liu Z, Xu H, Darrell T, Wang X (2021) Meta-baseline: exploring simple meta-learning for few-shot learning. In: Proceedings of the IEEE/CVF international conference on computer vision, pp 9062–9071

  28. Koch G, Zemel R, Salakhutdinov R et al (2015) Siamese neural networks for one-shot image recognition. In: ICML deep learning workshop, Lille, vol 2

  29. Snell J, Swersky K, Zemel R (2017) Prototypical networks for few-shot learning. In: Proceedings of the 31st international conference on neural information processing systems, pp 4080–4090

  30. Sung F, Yang Y, Zhang L, Xiang T, Torr PH, Hospedales TM (2018) Learning to compare: relation network for few-shot learning. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1199–1208

  31. Vinyals O, Blundell C, Lillicrap T, Wierstra D et al (2016) Matching networks for one shot learning. Adv Neural Inf Process Syst 29:3630–3638

    Google Scholar 

  32. Zhu K, Cao Y, Zhai W, Cheng J, Zha Z-J (2021) Self-promoted prototype refinement for few-shot class-incremental learning. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 6801–6810

  33. Zhang C, Song N, Lin G, Zheng Y, Pan P, Xu Y (2021) Few-shot incremental learning with continually evolved classifiers. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 12455–12464

  34. Chi Z, Gu L, Liu H, Wang Y, Yu Y, Tang J (2022) Metafscil: a meta-learning approach for few-shot class incremental learning. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 14166–14175

  35. Zhou D-W, Wang F-Y, Ye H-J, Ma L, Pu S, Zhan D-C (2022) Forward compatible few-shot class-incremental learning. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 9046–9056

  36. Huang H, Wu Z, Li W, Huo J, Gao Y (2021) Local descriptor-based multi-prototype network for few-shot learning. Pattern Recognit 116:107935

    Article  Google Scholar 

  37. Ji Z, Chai X, Yu Y, Zhang Z (2021) Reweighting and information-guidance networks for few-shot learning. Neurocomputing 423:13–23

    Article  Google Scholar 

  38. Deuschel J, Firmbach D, Geppert CI, Eckstein M, Hartmann A, Bruns V, Kuritcyn P, Dexl J, Hartmann D, Perrin D et al (2021) Multi-prototype few-shot learning in histopathology. In: Proceedings of the IEEE/CVF international conference on computer vision, pp 620–628

  39. Zimmermann RS, Sharma Y, Schneider S, Bethge M, Brendel W (2021) Contrastive learning inverts the data generating process. In: International conference on machine learning, PMLR, pp 12979–12990

  40. Zhong Z, Fini E, Roy S, Luo Z, Ricci E, Sebe N (2021) Neighborhood contrastive learning for novel class discovery. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 10867–10875

  41. Diba A, Sharma V, Safdari R, Lotfi D, Sarfraz S, Stiefelhagen R, Van Gool L (2021) Vi2clr: video and image for visual contrastive learning of representation. In: Proceedings of the IEEE/CVF international conference on computer vision, pp 1502–1512

  42. Russakovsky O, Deng J, Su H, Krause J, Satheesh S, Ma S, Huang Z, Karpathy A, Khosla A, Bernstein M et al (2015) Imagenet large scale visual recognition challenge. Int J Comput Vis 115(3):211–252

    Article  MathSciNet  Google Scholar 

  43. Wah C, Branson S, Welinder P, Perona P, Belongie S (2011) The caltech-ucsd birds-200-2011 dataset. Technical Report CNS-TR-2011-001, California Institute of Technology

  44. Zhang C, Cai Y, Lin G, Shen C (2020) Deepemd: few-shot image classification with differentiable earth mover’s distance and structured classifiers. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 12203–12213

  45. Rebuffi S-A, Kolesnikov A, Sperl G, Lampert C H (2017) ICARL: incremental classifier and representation learning. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 2001–2010

  46. Hou S, Pan X, Loy CC, Wang Z, Lin D, (2019) Learning a unified classifier incrementally via rebalancing. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 831–839

  47. Liu J, Song L, Qin Y, Prototype rectification for few-shot learning (2020). In: Computer vision–ECCV 2020: 16th European conference, Glasgow, UK, 23–28 Aug 2020, proceedings, part I 16, Springer, pp 741–756

  48. Chen Y, Wang X, Liu Z, Xu H, Darrell T (2020) A new meta-baseline for few-shot learning. arXiv:2003.04390

  49. Qi H, Brown M, Lowe DG (2018) Low-shot learning with imprinted weights. In: Proceedings of the IEEE conference on computer Cision and pattern recognition, pp 5822–5830

Download references

Acknowledgements

This work is supported by Shanghai Science and Technology Program “Federated based cross-domain and cross-task incremental learning” under Grant No. 21511100800, Shanghai Science and Technology Program ”Distributed and generative few-shot algorithm and theory research” under Grant No. 20511100600, Natural Science Foundation of China under Grant No. 62076094 and No. 62276098. Chinese Defense Program of Science and Technology under Grant No.2021-JCJQ-JJ-0041, China Aerospace Science and Technology Corporation Industry-University-Research Cooperation Foundation of the Eighth Research Institute under Grant No. SAST2021-007.

Author information

Authors and Affiliations

  1. Key Laboratory of Smart Manufacturing in Energy Chemical Process, Ministry of Education, East China University of Science and Technology, Shanghai, 200237, China

    Xinlei Xu, Zhe Wang, Zhiling Fu, Wei Guo, Ziqiu Chi & Dongdong Li

  2. Department of Computer Science and Engineering, East China University of Science and Technology, Shanghai, 200237, China

    Xinlei Xu, Zhe Wang, Zhiling Fu, Wei Guo, Ziqiu Chi & Dongdong Li

Authors
  1. Xinlei Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Zhe Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Zhiling Fu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Wei Guo
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Ziqiu Chi
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Dongdong Li
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Zhe Wang.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Xu, X., Wang, Z., Fu, Z. et al. Flexible few-shot class-incremental learning with prototype container. Neural Comput & Applic 35, 10875–10889 (2023). https://doi.org/10.1007/s00521-023-08272-y

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-023-08272-y

Keywords

Search

Navigation