Skip to main content
SpringerLink
Log in

Few-shot learning with deep balanced network and acceleration strategy

  • Original Article
  • Published:
International Journal of Machine Learning and Cybernetics Aims and scope Submit manuscript

Abstract

Deep networks are widely used in few-shot learning methods, but deep networks suffer from large-scale network parameters and computational effort. Aiming at the above problems, we present a novel few-shot learning method with deep balanced network and acceleration strategy. Firstly, a series of simple linear operations are applied to few original features to obtain the more features. More features are obtained with fewer parameters, thus reducing the network parameters and computational effort. Then the local cross-channel interaction mechanism without dimensionality reduction is used to further improve the performance with nearly no increase in parameters and computational effort, so as to obtain a deep balanced network to balance performance, parameters, and computational effort. Finally, an acceleration strategy is designed to solve the problem that the gradient update in the deep network takes a tremendous amount of time in new tasks, speeding up the adaptation process. The experimental results of traditional and fine-grained image classification show that the few-shot learning method with deep balanced network can achieve or even exceed the classification accuracy of some existing methods with fewer network parameters and computational effort. The cross-domain experiments further demonstrate the advantages of the method above the domain shift. Simultaneously, the time required for classification in new tasks can be significantly decreased by using the acceleration strategy.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

References

  1. Bi Z, Yu L, Gao H et al (2020) Improved vgg model-based efficient traffic sign recognition for safe driving in 5G scenarios. Int J Mach Learn Cybern. https://doi.org/10.1007/s13042-020-01185-5

    Article  Google Scholar 

  2. Chen B, Zhao T, Liu J et al (2021) Multipath feature recalibration densenet for image classification. Int J Mach Learn Cybern 12(3):651–660

    Article  Google Scholar 

  3. Fourati H, Maaloul R, Chaari L (2021) A survey of 5G network systems: challenges and machine learning approaches. Int J Mach Learn Cybern 12(2):385–431

    Article  Google Scholar 

  4. Mahindru A, Sangal AL (2021) Semidroid: a behavioral malware detector based on unsupervised machine learning techniques using feature selection approaches. Int J Mach Learn Cybern 12(5):1369–1411

    Article  Google Scholar 

  5. Jiang H, Zhan J, Sun B et al (2020) An MADM approach to covering-based variable precision fuzzy rough sets: an application to medical diagnosis. Int J Mach Learn Cybern 11(9):2181–2207

    Article  Google Scholar 

  6. Wang Y, Yao Q, Kwok J et al (2020) Generalizing from a few examples: a survey on few-shot learning. ACM Comput Surv 53(3):1–34

    Article  Google Scholar 

  7. Hospedales T, Antoniou A, Micaelli P et al (2020) Meta-learning in neural networks: a survey. arXiv:2004.05439

  8. Finn C, Abbeel P, Levine S (2017) Model-agnostic meta-learning for fast adaptation of deep networks. In: Proceedings of the International Conference on Machine Learning (ICML), Sydney, AUSTRALIA, pp 1126–1135

  9. Nichol A, Achiam J, Schulman J (2018) On first-order meta-learning algorithms. arXiv:1803.02999

  10. Vinyals O, Blundell C, Lillicrap T et al (2016) Matching networks for one shot learning. In: Proceedings of the Advances in Neural Information Processing Systems (NIPS), Barcelona, SPAIN, pp 3630–3638

  11. Snell J, Swersky K, Zemel RS (2017) Prototypical networks for few-shot learning. In: Proceedings of the Advances in Neural Information Processing Systems (NIPS), Long Beach, CA, USA, pp 4077–4087

  12. Sung F, Yang Y, Zhang L et al (2018) Learning to compare: relation network for few-shot learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Salt Lake City, UT, USA, pp 1199–1208

  13. Ravi S, Larochelle H (2017) Optimization as a model for few-shot learning. In: Proceedings of the International Conference on Learning Representations (ICLR), Toulon, FRANCE, https://openreview.net/forum?id=rJY0-Kcll

  14. Santoro A, Bartunov S, Botvinick M et al (2016) Meta-learning with memory-augmented neural networks. In: Proceedings of the International Conference on Machine Learning (ICML), New York City, NY, USA, pp 1842–1850

  15. Mishra N, Rohaninejad M, Chen X et al (2017) A simple neural attentive meta-learner. arXiv:1707.03141

  16. Munkhdalai T, Yu H (2017) Meta networks. In: Proceedings of the International Conference on Machine Learning (ICML), Sydney, AUSTRALIA, pp 2554–2563

  17. Li Z, Zhou F, Chen F et al (2017) Meta-sgd: learning to learn quickly for few-shot learning. arXiv:1707.09835

  18. Zhang R, Che T, Ghahramani Z et al (2018) Metagan: an adversarial approach to few-shot learning. In: Proceedings of the Advances in Neural Information Processing Systems (NIPS), Montréal, CANADA, pp 2371–2380

  19. Munkhdalai T, Yuan X, Mehri S et al (2018) Rapid adaptation with conditionally shifted neurons. In: Proceedings of the International Conference on Machine Learning (ICML), Stockholm, SWEDEN, pp 3661–3670

  20. Lee K, Maji S, Ravichandran A et al (2019) Meta-learning with differentiable convex optimization. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, USA, pp 10657–10665

  21. Wang X, Bao A, Cheng Y et al (2019) Weight-sharing multi-stage multi-scale ensemble convolutional neural network. Int J Mach Learn Cybern 10(7):1631–1642

    Article  Google Scholar 

  22. Sun Q, Liu Y , Chen Z et al (2019) Meta-transfer learning through hard tasks. arXiv:1910.03648

  23. Han K, Wang Y, Tian Q et al (2020) Ghostnet: more features from cheap operations. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), ELECTR NETWORK, pp 1577–1586

  24. Wang Q, Wu B, Zhu P et al (2020) Eca-net: efficient channel attention for deep convolutional neural networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), ELECTR NETWORK, pp 11531–11539

  25. Raghu A, Raghu M, Bengio S et al (2019) Rapid learning or feature reuse? towards understanding the effectiveness of maml. arXiv:1909.09157

  26. Bertinetto L, Henriques JF, Torr PHS et al (2018) Meta-learning with differentiable closed-form solvers. arXiv:1805.08136v3

  27. Oreshkin BN, Rodriguez P, Lacoste A (2018) Tadam: task dependent adaptive metric for improved few-shot learning. In: Proceedings of the Advances in Neural Information Processing Systems (NIPS), Montréal, CANADA, pp 721–731

  28. Rusu AA, Rao D, Sygnowski J et al (2018) Meta-learning with latent embedding optimization. arXiv:1807.05960

  29. Liu Y, Lee J, Park M et al (2018) Learning to propagate labels: transductive propagation network for few-shot learning. arXiv:1805.10002

  30. Franceschi L, Frasconi P, Salzo S et al (2018) Bilevel programming for hyperparameter optimization and meta-learning. In: Proceedings of the International Conference on Machine Learning (ICML), Stockholm, SWEDEN, pp 1568–1577

  31. Li W, Wang L, Xu J et al (2019) Revisiting local descriptor based image-to-class measure for few-shot learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, USA, pp 7253–7260

  32. Gidaris S, Komodakis N (2018) Dynamic few-shot visual learning without forgetting. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Salt Lake City, UT, USA, pp 4367–4375

  33. Chen W, Liu Y, Kira Z et al (2019) A closer look at few-shot classification. arXiv:1904.04232

  34. Bauer M, Rojas-Carulla M, Świątkowski JB, et al (2017) Discriminative k-shot learning using probabilistic models. arXiv:1706.00326.

  35. Li W, Wang L, Huo J et al (2020) Asymmetric distribution measure for few-shot learning. arXiv:2002.00153

  36. Yu Z, Raschka S (2020) Looking back to lower-level information in few-shot learning. arXiv:2005.13638

  37. Li S, Chen D, Liu B et al (2019) Memory-based neighbourhood embedding for visual recognition. In: Proceedings of the IEEE International Conference on Computer Vision (ICCV), Seoul, Koera (South), pp 6101–6110

  38. Ren M, Liao R, Fetaya E et al (2018) Incremental few-shot learning with attention attractor networks. arXiv:1810.07218v1

  39. Qiao L, Shi Y, Li J et al (2019) Transductive episodic-wise adaptive metric for few-shot learning. In: Proceedings of the IEEE International Conference on Computer Vision (ICCV), Seoul, Korea (South), pp 3602–3611.

  40. Ravichandran A, Bhotika R, Soatto S (2019) Few-shot learning with embedded class models and shot-free meta training. In: Proceedings of the IEEE International Conference on Computer Vision (ICCV), Seoul, Korea (South), pp 331–339

  41. Lifchitz Y, Avrithis Y, Picard S et al (2019) Dense classification and implanting for few-shot learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, USA, pp 9250–9259

  42. Dhillon GS, Chaudhari P, Ravichandran A et al (2019) A baseline for few-shot image classification. arXiv:1909.02729

  43. Patacchiola M, Turner J, Crowley EJ et al (2019) Bayesian meta-learning for the few-shot setting via deep kernels. arXiv:1910.05199

Download references

Funding

This work was supported by the National Natural Science Foundation of China under Grant 61772532 and Grant 61976215.

Author information

Authors and Affiliations

  1. Engineering Research Center of Intelligent Control for Underground Space, Ministry of Education, China University of Mining and Technology, Xuzhou, 221116, China

    Kang Wang, Xuesong Wang & Yuhu Cheng

  2. School of Information and Control Engineering, China University of Mining and Technology, Xuzhou, 221116, China

    Kang Wang, Xuesong Wang & Yuhu Cheng

  3. School of Computer Science and Engineering, South China University of Technology, Guangzhou, 510640, China

    Tong Zhang

Authors
  1. Kang Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xuesong Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Tong Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yuhu Cheng
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yuhu Cheng.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wang, K., Wang, X., Zhang, T. et al. Few-shot learning with deep balanced network and acceleration strategy. Int. J. Mach. Learn. & Cyber. 13, 133–144 (2022). https://doi.org/10.1007/s13042-021-01373-x

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s13042-021-01373-x

Keywords

Search

Navigation