Skip to main content
SpringerLink
Log in

Distributed few-shot learning with prototype distribution correction

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

Few-shot learning aims to learn a classifier that can perform well even if a few labeled samples are used for training. Many methods based on prototype models have recently been proposed and show good performance in few-shot learning. However, in the few-shot scenario, data is scarce. There is a deviation between the prototype calculated employing only a small amount of data in the support set and the actual prototype. Moreover, the features of the novel class extracted from the pre-trained model trained using base class data exhibit a bias. Additionally, the pre-trained model demonstrates high complexity and poor training efficiency. Therefore, we propose the Distributed Few-shot Learning with Prototype Distribution Correction to this end. Specifically, we employ pseudo-labels to fuse the sample features of the query set and modify the prototype to reduce the bias between the support and the query set. Then we utilize the features of the Gaussian distribution to transfer the base class features to the novel class to reduce the bias between novel classes and base classes. Finally, we combine distributed learning to improve the efficiency of the pre-trained model. We evaluate our method on Mini-ImageNet, Tiered-ImageNet, and CUB, three few-shot learning public datasets. The classification accuracy of our method improves in every case on every dataset and achieves state-of-the-art performance. In particular, the accuracy improves by 10.12% on 1-shot Mini-ImageNet.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

References

  1. Byerly A, Kalganova T, Ott R (2022) The current state of the art in deep learning for image classification: a review. In: Science and information conference, pp 88–105

  2. La Rosa B, Capobianco R, Nardi D (2023) A self-interpretable module for deep image classification on small data. Appl Intell 53(8):9115–9147

    Article  Google Scholar 

  3. Xu C, Gao W, Li T, Bai N, Li G, Zhang Y (2023) Teacher-student collaborative knowledge distillation for image classification. Appl Intell 53(2):1997–2009

    Article  Google Scholar 

  4. Song Y, Wang T, Cai P, Mondal SK, Sahoo JP (2023) A comprehensive survey of few-shot learning: evolution, applications, challenges, and opportunities. ACM Comput Surv

  5. Li X, Yang X, Ma Z, Xue J-H (2023) Deep metric learning for few-shot image classification: a review of recent developments. Pattern Recognit 109381

  6. Finn C, Abbeel P, Levine S (2017) Model-agnostic meta-learning for fast adaptation of deep networks. In: International conference on machine learning, PMLR pp 1126–1135

  7. Nichol A, Schulman J (2018) Reptile: a scalable metalearning algorithm 2(3),4. arXiv:1803.02999

  8. Abbas M, Xiao Q, Chen L, Chen P.-Y, Chen T (2022) Sharp-maml: sharpness-aware model-agnostic meta learning. In: International conference on machine learning, PMLR, pp 10–32

  9. Kang S, Hwang D, Eo M, Kim T, Rhee W (2023) Meta-learning with a geometry-adaptive preconditioner. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 16080–16090

  10. Satorras VG, Estrach JB (2018) Few-shot learning with graph neural networks. In: International conference on learning representations

  11. Kim J, Kim T, Kim S, Yoo CD (2019) Edge-labeling graph neural network for few-shot learning. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 11–20

  12. Snell J, Swersky K, Zemel R (2017) Prototypical networks for few-shot learning. In: Advances in neural information processing systems, pp 4080–4090

  13. Wertheimer D, Hariharan B (2019) Few-shot learning with localization in realistic settings. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 6558–6567

  14. Li W, Wang L, Huo J, Shi Y, Gao Y, Luo J (2021) Asymmetric distribution measure for few-shot learning. In: Proceedings of the international joint conference on artificial intelligence

  15. Li X, Wu J, Sun Z, Ma Z, Cao J, Xue J-H (2020) Bsnet: bi-similarity network for few-shot fine-grained image classification. IEEE Trans Image Process 30:1318–1331

    Article  MathSciNet  Google Scholar 

  16. Zhang C, Cai Y, Lin G, Shen C (2020) Deepemd: few-shot image classification with differentiable earth mover’s distance and structured classifiers. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 12203–12213

  17. Zhou Y, Guo Y, Hao S, Hong R (2022) Hierarchical prototype refinement with progressive inter-categorical discrimination maximization for few-shot learning. IEEE Trans Image Process 31:3414–3429

    Article  Google Scholar 

  18. Ma Y, Zhao S, Wang W, Li Y, King I (2022) Multimodality in meta-learning: a comprehensive survey. Knowl-Based Syst 250:108976

    Article  Google Scholar 

  19. Antoniou A, Edwards H, Storkey A (2019) How to train your maml. In: International conference on learning representations

  20. Zhang B, Li X, Ye Y, Huang Z, Zhang L (2021) Prototype completion with primitive knowledge for few-shot learning. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 3754–3762

  21. Allen K, Shelhamer E, Shin H, Tenenbaum J (2019) Infinite mixture prototypes for few-shot learning. In: International conference on machine learning, PMLR, pp 232–241

  22. Sergeev A, Balso MD (2018) Horovod: fast and easy distributed deep learning in tensorflow. CoRR aRxiv:1802.05799

  23. Assran M, Loizou N, Ballas N, Rabbat M (2019) Stochastic gradient push for distributed deep learning. In: International conference on machine learning, PMLR, pp 344–353

  24. Li S, Zhao Y, Varma R, Salpekar O, Noordhuis P, Li T, Paszke A, Smith J, Vaughan B, Damania P, Chintala S (2020) Pytorch distributed: Experiences on accelerating data parallel training. Proc VLDB Endow 13(12):3005–3018

    Article  Google Scholar 

  25. Vinyals O, Blundell C, Lillicrap T, Kavukcuoglu K, Wierstra D (2016) Matching networks for one shot learning. In: Advances in neural information processing systems, pp 3637–3645

  26. Mangla P, Kumari N, Sinha A, Singh M, Krishnamurthy B, Balasubramanian VN (2020) Charting the right manifold: Manifold mixup for few-shot learning. In: Proceedings of the IEEE/CVF winter conference on applications of computer vision, pp 2218–2227

  27. Sachin Ravi HL (2017) Optimization as a model for few-shot learning. In: International conference on learning representations

  28. Ren M, Triantafillou E, Ravi S, Snell J, Swersky K, Tenenbaum JB, Larochelle H, Zemel RS (2018) Meta-learning for semi-supervised few-shot classification

  29. Welinder P, Branson S, Mita T, Wah C, Schroff F, Belongie S, Perona P (2010) Caltech-ucsd birds 200

  30. Krizhevsky A, Sutskever I, Hinton GE (2012) Imagenet classification with deep convolutional neural networks. Advances in neural information processing systems 25:1097–1105

    Google Scholar 

  31. Sung F, Yang Y, Zhang L, Xiang T, Torr PH, Hospedales TM (2018) Learning to compare: Relation network for few-shot learning. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1199–1208

  32. Bertinetto L, Henriques J, Torr P, Vedaldi A (2019) Meta-learning with differentiable closed-form solvers

  33. Liu Y, Lee J, Park M, Kim S, Yang E, Hwang S, Yang Y (2019) Learning to propagate labels: Transductive propagation network for few-shot learning. In: International conference on learning representations

  34. Munkhdalai T, Yuan X, Mehri S, Trischler A () Rapid adaptation with conditionally shifted neurons. In: International conference on machine learning, PMLR, pp 3664–3673

  35. Zhang R, Che T, Ghahramani Z, Bengio Y, Song Y (2018) Metagan: an adversarial approach to few-shot learning. In: Advances in neural information processing systems, pp 2371–2380

  36. Jiang X, Havaei M, Varno F, Chartrand G, Chapados N, Matwin S (2018) Learning to learn with conditional class dependencies. In: International conference on learning representations

  37. Qiao S, Liu C, Shen W, Yuille AL (2018) Few-shot image recognition by predicting parameters from activations. In: Proceedings of the IEEE/CVF Conference on computer vision and pattern recognition, pp 7229–7238

  38. Rusu AA, Rao D, Sygnowski J, Vinyals O, Pascanu R, Osindero S, Hadsell R (2019) Meta-learning with latent embedding optimization

  39. Gidaris S, Bursuc A, Komodakis N, Pérez P, Cord M (2019) Boosting few-shot visual learning with self-supervision. In:Proceedings of the IEEE/CVF international conference on computer vision, pp 8059–8068

  40. Yang S, Liu L, Xu M (2021) Free lunch for few-shot learning: Distribution calibration. In: International conference on learning representations

  41. Liu J, Song L, Qin Y (2020) Prototype rectification for few-shot learning. In: European conference on computer vision, pp 741–756

  42. Russakovsky O, Deng J, Su H, Krause J, Satheesh S, Ma S, Huang Z, Karpathy A, Khosla A, Bernstein M et al (2015) Imagenet large scale visual recognition challenge. Int J Comput Vision 115(3):211–252

    Article  MathSciNet  Google Scholar 

  43. Pedregosa F, Varoquaux G, Gramfort A, Michel V, Thirion B, Grisel O, Blondel M, Prettenhofer P, Weiss R, Dubourg V et al (2011) Scikit-learn: Machine learning in python. J Mach Learn Res 12:2825–2830

    MathSciNet  Google Scholar 

  44. Gidaris S, Komodakis N (2018) Dynamic few-shot visual learning without forgetting. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 4367–4375

  45. Li H, Eigen D, Dodge S, Zeiler M, Wang X (2019) Finding task-relevant features for few-shot learning by category traversal. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 1–10

  46. Yoon SW, Seo J, Moon J (2019) Tapnet: neural network augmented with task-adaptive projection for few-shot learning. In: International conference on machine learning, PMLR, pp 7115–7123

  47. Sun Q, Liu Y, Chua T-S, Schiele B (2019) Meta-transfer learning for few-shot learning. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 403–412

  48. Lee K, Maji S, Ravichandran A, Soatto S (2019) Meta-learning with differentiable convex optimization. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 10657–10665

  49. Ravichandran A, Bhotika R, Soatto S (2019) Few-shot learning with embedded class models and shot-free meta training. In: Proceedings of the IEEE/CVF international conference on computer vision, pp 331–339

  50. Wang Z, Cao C, Zhu Y (2020) Entropy and confidence-based undersampling boosting random forests for imbalanced problems. IEEE Trans Neural Netw Learn Syst 31(12):5178–5191

    Article  Google Scholar 

  51. Chen W-Y, Liu Y-C, Kira Z, Wang Y-CF, Huang J-B (2019) A closer look at few-shot classification. In: International conference on learning representations

  52. Li W, Wang L, Xu J, Huo J, Gao Y, Luo J (2019) Revisiting local descriptor based image-to-class measure for few-shot learning. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 7260–7268

  53. Ye H, Hu H, Zhan D, Sha F (2018) Learning embedding adaptation for few-shot learning. CoRR arXiv:1812.03664

  54. Ziko I, Dolz J, Granger E, Ayed IB (2020) Laplacian regularized few-shot learning. In: International conference on machine learning, PMLR, pp 11660–11670

  55. Wang Y, Chao W, Weinberger KQ, Maaten L (2019) Simpleshot: revisiting nearest-neighbor classification for few-shot learning. CoRR arXiv:1911.04623

Download references

Acknowledgements

This work is supported by Shanghai Science and Technology Program “Distributed and generative few-shot algorithm and theory research” under Grant No. 20511100600, Natural Science Foundation of China under Grant No. 62076094, Shanghai Science and Technology Program “Federated based cross-domain and cross-task incremental learning” under Grant No. 21511100800, Chinese Defense Program of Science and Technology under Grant No.2021-JCJQ-JJ-0041, China Aerospace Science and Technology Corporation Industry-University-Research Cooperation Foundation of the Eighth Research Institute under Grant No.SAST2021-007.

Author information

Author notes

  1. Zhiling Fu and Dongfang Tang contributed equally to this work.

Authors and Affiliations

  1. Department of Computer Science and Engineering, East China University of Science and Technology, Shanghai, 200237, People’s Republic of China

    Zhiling Fu, Pingchuan Ma & Zhe Wang

  2. Huadong Hospital Affiliated to Fudan University, Jing’An, 200031, China

    Dongfang Tang & Wen Gao

Authors
  1. Zhiling Fu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Dongfang Tang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Pingchuan Ma
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Zhe Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Wen Gao
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Zhe Wang.

Ethics declarations

Conflicts of interest

The authors have declared no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Fu, Z., Tang, D., Ma, P. et al. Distributed few-shot learning with prototype distribution correction. Appl Intell 53, 30552–30565 (2023). https://doi.org/10.1007/s10489-023-05103-4

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-023-05103-4

Keywords

Search

Navigation