Skip to main content
SpringerLink
Log in

Convolutional Shrinkage Neural Networks Based Model-Agnostic Meta-Learning for Few-Shot Learning

  • Published:
Neural Processing Letters Aims and scope Submit manuscript

Abstract

Meta Learning (ML) has the ability to quickly learn from a small number of samples, and has become an important research field after reinforcement learning. However, the complexity of sample features severely reduces the performance of few-shot learning, and proper feature selection plays a vital role in the performance of neural networks. To address this problem, this article draws up a new type of convolutional neural network with an attention mechanism, namely, convolutional shrinkage neural networks (CSNNs), using the characteristics of negligible noise to obtain a good optimization parameter model. Moreover, soft thresholding is inserted into the network architectures as nonlinear transformation layers to eliminate nonessential features. In addition, considering that it is difficult to set appropriate values for the thresholds, the developed convolutional shrinkage neural networks integrates some specialized neural networks into trainable modules to automatically set the thresholds. To illustrate the effectiveness of the proposed method, the model-agnostic meta-learning method is considered for testing. The results show that the improved method can significantly improve the accuracy of few-shot images classification and enhance the generalization performance.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

References

  1. Tang WX, Li B, Barni M et al (2021) An automatic cost learning framework for image steganography using deep reinforcement learning. IEEE Trans Inf Forensics Secur 16:952–967

    Article  Google Scholar 

  2. Choi Y, Lee K, Oh S (2019) Distributional deep reinforcement learning with a mixture of Gaussians. In: Proceedings - IEEE International Conference on Robotics and Automation 2019-May: p 9791–9797

  3. Zoph B, Vasudevan V, Shlens J et al (2018) Learning Transferable Architectures for Scalable Image Recognition. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, p 8697–8710

  4. Baker B, Gupta O, Naik N et al (2017) Designing neural network architectures using reinforcement learning. In: 5th International Conference on Learning Representations, ICLR 2017 - Conference Track Proceedings

  5. Zoph B, Le QV (2017) Neural architecture search with reinforcement learning. In: 5th International Conference on Learning Representations, ICLR 2017 - Conference Track Proceedings

  6. Pham H, Guan M Y, Zoph B et al (2018) Efficient Neural Architecture Search via parameter Sharing. In: 35th International Conference on Machine Learning, ICML 2018, vol 9, p 6522–6531

  7. Pang G, Shen C, Cao L et al (2021) Deep Learning for Anomaly Detection: A Review. ACM Comput Surv 54(2):1–38

    Article  Google Scholar 

  8. Leake D, Crandall D (2020) On Bringing Case-Based Reasoning Methodology to Deep Learning. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 12311 LNAI: 343–348

  9. Lu H, Jin L, Luo X et al (2019) RNN for Solving Perturbed Time-Varying Underdetermined Linear System With Double Bound Limits on Residual Errors and State Variables. IEEE Trans Ind Inform 15(11):5931–5942

    Article  Google Scholar 

  10. Hong Y, Niu L, Zhang J et al (2020) Matchinggan: Matching-Based Few-Shot Image Generation. IEEE Int Conf Multimed Expo (ICME) 2020:1–6

    Google Scholar 

  11. Lake B, Salakhutdinov R, Tenenbaum J (2015) Human-level concept learning through probabilistic program induction. Science 350(6266):1332–1338

    Article  MathSciNet  MATH  Google Scholar 

  12. Wang J, Zhai Y (2020) Prototypical Siamese Networks for Few-shot Learning. In: 2020 IEEE 10th International Conference on Electronics Information and Emergency Communication (ICEIEC), p 178–181

  13. Das D, Lee CSG (2020) A Two-Stage Approach to Few-Shot Learning for Image Recognition. IEEE Trans Image Process 29:3336–3350

    Article  MATH  Google Scholar 

  14. Ramalho T, Garnelo M (2019) Adaptive Posterior Learning: few-shot learning with a surprise-based memory module. In: 7th International Conference on Learning Representations, ICLR

  15. Finn C, Abbeel P, Levine S (2017) Model-agnostic meta-learning for fast adaptation of deep networks. In: 34th International Conference on Machine Learning, ICML 2017, vol 3, p 1856–1868

  16. Antoniou A, Storkey A, Edwards H (2019) How to train your MAML. In: 7th International Conference on Learning Representations, ICLR

  17. Liu Y, Lee J, Park M et al (2019) Learning to propagate labels: Transductive propagation network for few-shot learning. In: 7th International Conference on Learning Representations, ICLR

  18. Yao H, Wei Y, Huang J et al (2019) Hierarchically structured meta-learning. In: 36th International Conference on Machine Learning, ICML, 2019-June, p 12189–12209

  19. Yao H, Wu X, Tao Z et al (2020) Automated relational meta-learning. arXiv:2001.00745v1

  20. Ravi S, Larochelle H (2017) Optimization as a model for few-shot learning. In: 5th International Conference on Learning Representations, ICLR 2017 - Conference Track Proceedings

  21. Santoro A, Bartunov S, Botvinick M et al (2016) Meta-Learning with Memory-Augmented Neural Networks. In: 33rd International Conference on Machine Learning, ICML 2016, vol 4, p 2740–2751

  22. Finn C, Rajeswaran A, Kakade S et al (2019) Online meta-learning. In: 36th International Conference on Machine Learning, ICML 2019, 2019-June, p 3398–3410

  23. Munkhdalai T, Yu H (2017) Meta networks. In: 34th International Conference on Machine Learning, ICML 2017, vol 5, p 3933–3943

  24. Lee K, Maji S, Ravichandran A et al (2019) Meta-learning with differentiable convex optimization. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, p 10649-10657

  25. Xue T, Yu H (2020) Model-Agnostic Metalearning-Based Text-Driven Visual Navigation Model for Unfamiliar Tasks. IEEE Access 8:166742–166752

    Article  Google Scholar 

  26. Isogawa K, Ida T, Shiodera T et al (2018) Deep shrinkage convolutional neural network for adaptive noise reduction. IEEE Signal Process Lett 25:224–228

    Article  Google Scholar 

  27. Zhao M, Zhong S, Fu X et al (2020) Deep Residual Shrinkage Networks for Fault Diagnosis. IEEE Trans Industr Inf 16:4681–4690

    Article  Google Scholar 

  28. Hu J, Shen L, Albanie S et al (2020) Squeeze-and-Excitation Networks. IEEE Trans Pattern Anal Mach Intell 42:2011–2023

    Article  Google Scholar 

  29. Tan M, Chen B, Pang R et al (2019) Mnasnet: Platform-aware neural architecture search for mobile. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2019-June, p 2815–2823

  30. Vaswani A, Shazeer N, Parmar N et al (2017) Attention is all you need. Advances in Neural Information Processing Systems, 2017-December: 5999-6009

  31. Gu J, Wang Z, Kuen J et al (2018) Recent Advances in Convolutional Neural Networks. Pattern Recogn 77:354–377

    Article  Google Scholar 

  32. Zhao ZQ, Zheng P, Xu S-T et al (2019) Object Detection with Deep Learning: A Review. IEEE Trans Neural Netw Learn Syst 30:3212–3232

    Article  Google Scholar 

  33. Yu SD, Liu LL, Wang ZY et al (2019) Transferring deep neural networks for the differentiation of mammographic breast lesions. Sci China Tech Sci 62:441–447

    Article  Google Scholar 

  34. Nichol A, Schulman J Reptile: a scalable metalearning algorithm. arXiv:1803.02999v1

  35. Munkhdalai T, Yuan X, Mehri S et al (2018) Rapid adaptation with conditionallyshifted neurons. In: 35th International Conference on Machine Learning, ICML, vol 8, p 5898–5909

  36. Das D, LeeC (2020) A Two-Stage Approach to Few-Shot Learning for Image Recognition. IEEE Trans Image Process 29:3336–3350

    Article  MATH  Google Scholar 

  37. Vinyals O, Blundell C, Lillicrap T et al (2016) Matching networks for one shot learning. In: 30th conference on neural information processing systems (NIPS), vol 29

  38. Wang R, Zhang X, Liu C (2021) Meta-Prototypical Learning for Domain-Agnostic Few-Shot Recognition. IEEE Transactions on Neural Networks and Learning Systems, 1–7

  39. Lee H, Lee H, Na D et al (2020) Learning to Balance: Bayesian Meta-Learning for Imbalanced and Out-of-distribution Tasks. arXiv:1905.12917, 1–15

  40. Luo X, Zhou M, Shang M et al (2016) A Novel Approach to Extracting Non-Negative Latent Factors From Non-Negative Big Sparse Matrices. IEEE Access 4:2649–2655

    Article  Google Scholar 

  41. Luo X, Zhou M, Li S et al (2021) Algorithms of Unconstrained Non-Negative Latent Factor Analysis for Recommender Systems. IEEE Trans Big Data 7(1):227–240

    Article  Google Scholar 

Download references

Acknowledgements

This work was supported in part by the National Key Research and Development Program of China under Grant 2018YFB1700200, in part by the National Natural Science Foundation of China under Grant U1908212, 61533015 and 92067205, in part by the State Key Laboratory of Robotics of China under Grant Y91Z081, also in part by the Natural Science Foundation of Liaoning Province under Grant 2020-KF-11-02.

Author information

Authors and Affiliations

  1. State Key Laboratory of Robotics, Shenyang Institute of Automation, Chinese Academy of Sciences, 110016, Shenyang, China

    Yunpeng He, Peng Zeng, Qingwei Dong, Ding Liu & Yuqi Liu

  2. Key Laboratory of Networked Control Systems, Chinese Academy of Sciences, 110016, Shenyang, China

    Yunpeng He, Peng Zeng, Qingwei Dong, Ding Liu & Yuqi Liu

  3. Institutes for Robotics and Intelligent Manufacturing, Chinese Academy of Sciences, 110169, Shenyang, China

    Yunpeng He, Peng Zeng, Qingwei Dong, Ding Liu & Yuqi Liu

  4. University of Chinese Academy of Sciences, 100049, Beijing, China

    Yunpeng He, Qingwei Dong & Ding Liu

  5. Shenyang University of Technology, 110870, Shenyang, China

    Chuanzhi Zang

Authors
  1. Yunpeng He
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Chuanzhi Zang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Peng Zeng
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Qingwei Dong
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Ding Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Yuqi Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Peng Zeng.

Ethics declarations

Conflicts of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

He, Y., Zang, C., Zeng, P. et al. Convolutional Shrinkage Neural Networks Based Model-Agnostic Meta-Learning for Few-Shot Learning. Neural Process Lett 55, 505–518 (2023). https://doi.org/10.1007/s11063-022-10894-7

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11063-022-10894-7

Keywords

Search

Navigation