Skip to main content
SpringerLink
Log in

Hierarchical few-shot learning based on coarse- and fine-grained relation network

  • Published:
Artificial Intelligence Review Aims and scope Submit manuscript

Abstract

Few-shot learning plays an important role in the field of machine learning. Many existing methods based on relation network achieve satisfactory results. However, these methods assume that classes are independent of each other and ignore their relationship. In this paper, we propose a hierarchical few-shot learning model based on coarse- and fine-grained relation network (HCRN), which constructs a hierarchical structure by mining the relationship among different classes. Firstly, we extract deep and shallow features from different layers at a convolutional neural network. The shallow feature information contains more common features among similar classes, while the deep feature information is more specific. The complementary of these different types of data features can effectively construct coarse- and fine-grained structures by clustering. Secondly, we design coarse- and fine-grained relation networks to classify according to the guidance of the hierarchical structure. The hierarchical class structure learned from data is important auxiliary information for classification. Experimental results show that HCRN can outperform several state-of-the-art models on the Omniglot and miniImageNet datasets. Especially, HCRN obtains 6.47% improvement over the next best under the 5-way 1-shot setting on the miniImageNet dataset.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

References

  • Alfassy A, Karlinsky L, Aides A, Shtok J, Harary S, Feris R, Giryes R, Bronstein AM (2019) Laso: label-set operations networks for multi-label few-shot learning. In: IEEE Conference on Computer Vision and Pattern Recognition, pp 6548–6557

  • Altae-Tran H, Ramsundar B, Pappu AS, Pande V (2017) Low data drug discovery with one-shot learning. ACS Cent Sci 3(4):283–293

    Article  Google Scholar 

  • Asiri N, Bchir O, Ismail MMB, Zakariah M, Alotaibi YA (2021) Image-based smoke detection using feature mapping and discrimination. Soft Comput 25(5):3665–3674

    Article  Google Scholar 

  • Bateni P, Barber J, Meent JWVD, Wood F (2020) Improving few-shot visual classification with unlabelled examples. In: IEEE Conference on Computer Vision and Pattern Recognition, pp 14481–14490

  • Bateni P, Goyal R, Masrani V, Wood F, Sigal L (2020) Improved few-shot visual classification. In: IEEE Conference on Computer Vision and Pattern Recognition, pp 14493–14502

  • Bertinetto L, Henriques JF, Valmadre J, Torr PH, Vedaldi A (2016) Learning feed-forward one-shot learners. In: International Conference on Neural Information Processing Systems, pp 523–531

  • Chen D, Lv J, Yi Z (2017) Unsupervised multi-manifold clustering by learning deep representation. In: AAAI Conference on Artificial Intelligence

  • Cheng C, Li C, Han Y, Zhu Y (2021) A semi-supervised deep learning image caption model based on pseudo label and n-gram. Int J Approx Reason 131:93–107

    Article  MathSciNet  MATH  Google Scholar 

  • Cui Y, Liao Q, Hu D, An W, Liu L (2022) Coarse-to-fine pseudo supervision guided meta-task optimization for few-shot object classification. Pattern Recogn 122:108296

    Article  Google Scholar 

  • Do K, Tran T, Venkatesh S (2021) Clustering by maximizing mutual information across views. In: IEEE/CVF International Conference on Computer Vision, pp 9928–9938

  • Edwards H, Storkey A (2017) Towards a neural statistician. In: International Conference on Learning Representations

  • Feifei L, Fergus R, Perona P (2006) One-shot learning of object categories. IEEE Trans Pattern Anal Mach Intell 28(4):594–611

    Article  Google Scholar 

  • Fink M (2005) Object classification from a single example utilizing class relevance metrics. In: International Conference on Neural Information Processing Systems, pp 449–456

  • Finn C, Abbeel P, Levine S (2017) Model-agnostic meta-learning for fast adaptation of deep networks. In: International Conference on Machine Learning, pp 1126–1135

  • Fish E, Weinbren J, Gilbert A (2021) Rethinking genre classification with fine grained semantic clustering. In: IEEE International Conference on Image Processing, pp 1274–1278

  • Gidaris S, Komodakis N (2018) Dynamic few-shot visual learning without forgetting. In: IEEE Conference on Computer Vision and Pattern Recognition, pp 4367–4375

  • He J, Hong R, Liu X, Xu M, Zha Z, Wang M (2020) Memory-augmented relation network for few-shot learning. In: ACM International Conference on Multimedia, pp 1236–1244

  • Hui B, Zhu P, Hu Q, Wang Q (2019) Self-attention relation network for few-shot learning. In: IEEE International Conference on Multimedia & Expo Workshops, pp 198–203

  • Ji P, Zhang T, Li H, Salzmann M, Reid I (2017) Deep subspace clustering networks. In: International Conference on Neural Information Processing Systems, pp 23–32

  • Jiang S, Wu G (2021) Mrn: moment relation network for natural language video localization with transfer learning. Int J Pattern Recogn Artif Intell, page 2152009

  • Kaiser L, Nachum O, Roy A, Bengio S (2017) Learning to remember rare events. arXiv preprint arXiv:1703.03129

  • Kingma D, Ba J (2015) Adam: a method for stochastic optimization. In: International Conference on Learning Representations

  • Koch G, Zemel R, Salakhutdinov R (2015) Siamese neural networks for one-shot image recognition. In: ICML Deep Learning Workshop

  • Lake BM, Salakhutdinov R, Gross J, Tenenbaum JB (2011) One shot learning of simple visual concepts. In: Annual Meeting of the Cognitive Science Society

  • Larochelle S (2016) Optimization as a model for few-shot learning. arXiv preprint arXiv:1606.04080

  • Li Z, Nie F, Chang X, Nie L, Zhang H, Yang Y (2018) Rank-constrained spectral clustering with flexible embedding. IEEE Trans Neural Networks Learn Syst 29(12):6073–6082

    Article  MathSciNet  Google Scholar 

  • Li Z, Nie F, Chang X, Yang Y, Zhang C, Sebe N (2018) Dynamic affinity graph construction for spectral clustering using multiple features. IEEE Trans Neural Networks Learn Syst 29(12):6323–6332

    Article  MathSciNet  Google Scholar 

  • Li Z, Yao L, Chang X, Zhan K, Sun J, Zhang H (2019) Zero-shot event detection via event-adaptive concept relevance mining. Pattern Recogn 88:595–603

    Article  Google Scholar 

  • Liu B, Yu X, Yu A, Zhang P, Wan G, Wang R (2018) Deep few-shot learning for hyperspectral image classification. IEEE Trans Geosci Remote Sens 57(4):2290–2304

    Article  Google Scholar 

  • Liu Y, Schiele B, Sun Q (2020) An ensemble of epoch-wise empirical bayes for few-shot learning. In: European Conference on Computer Vision, pp 404–421

  • Miller EG, Matsakis NE, Viola PA (2000) Learning from one example through shared densities on transforms. In: IEEE Conference on Computer Vision and Pattern Recognition, pp 464–471

  • Munkhdalai T, Yu H (2017) Meta networks. In: International Conferencerning, pp 2554–2563

  • Oreshkin BN, Rodriguez P, Lacoste A (2018) Tadam: Task dependent adaptive metric for improved few-shot learning. In: International Conference on Neural Information Processing Systems, pp 719–729

  • Qiu Z, Zhao H (2022) A fuzzy rough set approach to hierarchical feature selection based on hausdorff distance. Applied Intelligence, pp 1–14

  • Rahbar M, Yazdani S (2021) Historical knowledge-based mbo for global optimization problems and its application to clustering optimization. Soft Comput 25(5):3485–3501

    Article  Google Scholar 

  • Rusu AA, Rao D, Sygnowski J, Vinyals O, Pascanu R, Osindero S, Hadsell R (2019) Meta-learning with latent embedding optimization. In: International Conference on Learning Representations

  • Santoro A, Bartunov S, Botvinick M, Wierstra D, Lillicrap T (2016) Meta-learning with memory-augmented neural networks. In: International Conference on Machine Learning, pp 1842–1850

  • Selvaraju RR, Cogswell M, Das A, Vedantam R, Parikh D, Batra D (2020) Grad-cam: visual explanations from deep networks via gradient-based localization. Int J Comput Vision 128(2):336–359

    Article  Google Scholar 

  • Seo J, Yoon SW, Moon J (2020) Task-adaptive clustering for semi-supervised few-shot classification. arXiv preprint arXiv:2003.08221

  • Shyam P, Gupta S, Dukkipati A (2017) Attentive recurrent comparators. In: International Conference on Machine Learning, pp 3173–3181

  • Snell J, Swersky K, Zemel RS (2017) Prototypical networks for few-shot learning. In: International Conference on Neural Information Processing Systems, pp 4080–4090

  • Sung F, Yang Y, Zhang L, Xiang T, Torr PH, Hospedales TM (2018) Learning to compare: relation network for few-shot learning. In: IEEE Conference on Computer Vision and Pattern Recognition, pp 1199–1208

  • Tang KD, Tappen MF, Sukthankar R, Lampert CH (2010) Optimizing one-shot recognition with micro-set learning. In: IEEE Conference on Computer Vision and Pattern Recognition, pp 3027–3034

  • Triantafillou E, Zemel RS, Urtasun R (2017) Few-shot learning through an information retrieval lens. In: International Conference on Neural Information Processing Systems, pp 2255–2265

  • Vinyals O, Blundell C, Lillicrap T, Kavukcuoglu K, Wierstra D (2016) Matching networks for one shot learning. In: International Conference on Neural Information Processing Systems, pp 3637–3645

  • Wang X, Zhang Q, Jiang C, Zhang Y (2020) Coarse-to-fine grained image splicing localization method based on noise level inconsistency. In: International Conference on Computing, Networking and Communications, pp 79–83. IEEE

  • Wang Y, Hu Q, Chen H, Qian Y (2022) Uncertainty instructed multi-granularity decision for large-scale hierarchical classification. Inf Sci 586:644–661

    Article  Google Scholar 

  • Wang Y, Liu R, Lin D, Chen D, Li P, Hu Q, Chen CLP (2021) Coarse-to-fine: progressive knowledge transfer-based multitask convolutional neural network for intelligent large-scale fault diagnosis. In: IEEE Transactions on Neural Networks and Learning Systems, pp 1–14

  • Wang Z, Miao Z, Zhen X, Qiu Q (2021) Learning to learn dense gaussian processes for few-shot learning. Neural Information Processing Systems, 34

  • Wen W, Liu Y, Ouyang C, Lin Q, Chung T (2021) Enhanced prototypical network for few-shot relation extraction. Inf Process Manag 58(4):102596

    Article  Google Scholar 

  • Xing EP, Ng AY, Jordan MI, Russell SJ (2002) Distance metric learning with application to clustering with side-information. In: International Conference on Neural Information Processing Systems, pp 505–512

  • Xu Z, Yu F, Liu C, Wu Z, Wang H, Chen X (2022) Falcon: fine-grained feature map sparsity computing with decomposed convolutions for inference optimization. In: IEEE/CVF Winter Conference on Applications of Computer Vision, pp 350–360

  • Yang B, Fu X, Sidiropoulos ND, Hong M (2017) Towards k-means-friendly spaces: simultaneous deep learning and clustering. In: International Conference on Machine Learning, pp 3861–3870

  • Yang P, Ren S, Zhao Y, Li P (2022) Calibrating cnns for few-shot meta learning. In: IEEE/CVF Winter Conference on Applications of Computer Vision, pp 2090–2099

  • Yao Y (2000) Granular computing: basic issues and possible solutions. In: Joint Conference on Information Sciences, vol 1, pp 186–189

  • Yao Y (2004) A partition model of granular computing. Trans Rough Sets I(2):232–253

    MATH  Google Scholar 

  • Yao Y (2011) Artificial intelligence perspectives on granular computing. In Granular Computing and Intelligent Systems, pages 17–34. Springer

  • Yao Y (2016) A triarchic theory of granular computing. Granular Comput 1(2):145–157

    Article  Google Scholar 

  • Yu X, Liu H, Wu Y, Zhang C (2021) Fine-grained similarity fusion for multi-view spectral clustering. Inf Sci 568:350–368

    Article  MathSciNet  Google Scholar 

  • Zadeh LA (1997) Toward a theory of fuzzy information granulation and its centrality in human reasoning and fuzzy logic. Fuzzy Sets Syst 90(2):111–127

    Article  MathSciNet  MATH  Google Scholar 

  • Zhan K, Zhang C, Guan J, Wang J (2017) Graph learning for multiview clustering. IEEE Trans Cybern 48(10):2887–2895

    Article  Google Scholar 

  • Zhang C, Hu Q, Fu H, Zhu P, Cao X (2017) Latent multi-view subspace clustering. In: IEEE Conference on Computer Vision and Pattern Recognition, pp 4279–4287

  • Zhang H, Zhu J, Chen J, Liu J, Ji L (2021) Zero-shot fine-grained entity typing in information security based on ontology. Knowl-Based Syst 232:107472

    Article  Google Scholar 

  • Zhang R, Song X, Ying S, Ren H, Zhang B, Wang H (2021) Ca-csm: a novel clustering algorithm based on cluster center selection model. Soft Comput 25(13):8015–8033

    Article  Google Scholar 

  • Zhao H, Hu Q, Zhu P, Wang Y, Wang P (2021) A recursive regularization based feature selection framework for hierarchical classification. IEEE Trans Knowl Data Eng 33(7):2833–2846

    Article  Google Scholar 

  • Zhao H, Wang P, Hu Q, Zhu P (2019) Fuzzy rough set based feature selection for large-scale hierarchical classification. IEEE Trans Fuzzy Syst 27(10):1891–1903

    Article  Google Scholar 

  • Zheng Q, Zhu J, Li Z, Pang S, Wang J, Li Y (2020) Feature concatenation multi-view subspace clustering. Neurocomputing 379:89–102

    Article  Google Scholar 

  • Zhou S, Deng C, Piao Z, Zhao B (2020) Few-shot traffic sign recognition with clustering inductive bias and random neural network. Pattern Recogn 100:107160

    Article  Google Scholar 

Download references

Acknowledgements

This work was supported by the National Natural Science Foundation of China under Grant No. 62141602 and the Natural Science Foundation of Fujian Province under Grant No. 2021J011003.

Funding

Existing funding was provided in acknowledgment.

Author information

Authors and Affiliations

  1. School of Computer Science, Minnan Normal University, Zhangzhou, 363000, Fujian, China

    Zhiping Wu & Hong Zhao

  2. Key Laboratory of Data Science and Intelligence Application, Fujian Province University, Zhangzhou, 363000, Fujian, China

    Zhiping Wu & Hong Zhao

Authors
  1. Zhiping Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Hong Zhao
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

Methodology, software, data curation, and writing-original draft were done by Zhiping Wu. Validation, methodology, reviewing, and editing were done by Hong Zhao.

Corresponding author

Correspondence to Hong Zhao.

Ethics declarations

Conflict of interest

All authors declare that there is no conflict of interest in this manuscript.

Ethical approval

This paper does not contain any studies with human participants performed by any of the authors. This material is the authors’ original work, which has not been previously published elsewhere. The paper is not currently being considered for publication elsewhere. The paper reflects the authors’ research and analysis in a truthful and complete manner.

Informed consent

Informed consent was obtained from all individual participants included in the study.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wu, Z., Zhao, H. Hierarchical few-shot learning based on coarse- and fine-grained relation network. Artif Intell Rev 56, 2011–2030 (2023). https://doi.org/10.1007/s10462-022-10223-3

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10462-022-10223-3

Keywords

Search

Navigation