Skip to main content
SpringerLink
Log in

Dual-model Collaborative Learning with Knowledge Clustering for Few-shot Image Classification

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Few-shot learning (FSL) refers to adapt model to novel classes with few annotations. Existing methods generally utilize a single model’s information directly extracted from samples. Extra information are helpful to enhance the generation of the model. This paper focuses on designing a dual-model structure to learn the correlation between two models and introduce the center loss to cluster the same sort of samples and enhance the representation of samples. The center loss is to improve the generalization of the active branch. Moreover, we combine meta-learning. The meta-training has multiple tasks, and each task has two stages in our work, which firstly trains one model with soft labels from another fixed model and center loss. The optimal predictions of the active model are close to the soft and actual labels. Meanwhile, the same samples will gather together, attempting to minimize the intra-class differences. It could enhance the generalization and robustness of the model. We conduct experiments on miniImageNet, tieredImageNet and CUB. The results show the excellence of our proposed method.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

Data availability

All data generated or analyzed during this study are included in this paper. The data of this work is available on request from the authors. The data of MiniImageNet tieredImageNet, and CUB are also available in the public repository.

References

  1. Bhatti UA, Yu Z, Hasnain A, Nawaz SA, Yuan L, Wen L, Bhatti MA (2022) Evaluating the impact of roads on the diversity pattern and density of trees to improve the conservation of species. Envi Sci Pollut Res 1–11

  2. Bhatti UA, Yuan L, Yu Z, Nawaz SA, Mehmood A, Bhatti MA, Nizamani MM, Xiao S et al (2021) Predictive data modeling using sp-knn for risk factor evaluation in urban demographical healthcare data. J Med Imaging Health Inform 11(1):7–14

    Article  Google Scholar 

  3. Bhatti UA, Yu Z, Li J, Nawaz SA, Mehmood A, Zhang K, Yuan L (2020) Hybrid watermarking algorithm using clifford algebra with arnold scrambling and chaotic encryption. IEEE Access 8:76386–76398

    Article  Google Scholar 

  4. Bhatti UA, Yu Z, Yuan L, Nawaz SA, Aamir M, Bhatti MA (2022) A robust remote sensing image watermarking algorithm based on region-specific surf. In: Proceedings of International Conference on Information Technology and Applications: ICITA 2021, pp. 75–85. Springer

  5. Bhatti UA, Yu Z, Yuan L, Zeeshan Z, Nawaz SA, Bhatti M, Mehmood A, Ain QU, Wen L (2020) Geometric algebra applications in geospatial artificial intelligence and remote sensing image processing. IEEE Access 8:155783–155796

    Article  Google Scholar 

  6. Wu Y, Liu H, Fu Y (2017) Low-shot face recognition with hybrid classifiers. In: Proceedings of the IEEE International Conference on Computer Vision (ICCV) Workshops

  7. Mahajan K, Sharma M, Vig L (2020) Metadermdiagnosis: Few-shot skin disease identification using meta-learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 730–731

  8. Chen Y, Peng X, Kong L, Dong G, Remani A, Leach R (2021) Defect inspection technologies for additive manufacturing. Int J Extreme Manuf 3(2):022002. https://doi.org/10.1088/2631-7990/abe0d0

    Article  CAS  Google Scholar 

  9. Chen C, Yang X, Xu C, Huang X, Ma Z (2021) Eckpn: Explicit class knowledge propagation network for transductive few-shot learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 6596–6605

  10. Snell J, Swersky K, Zemel R (2017) Prototypical networks for few-shot learning. In: Guyon I, Luxburg UV, Bengio S, Wallach H, Fergus R, Vishwanathan S, Garnett R (Eds.) Advances in Neural Information Processing Systems, vol. 30. Curran Associates, Inc., ???

  11. Chen Y, Wang X, Liu Z, Xu H, Darrell T (2020) A new meta-baseline for few-shot learning

  12. Weinberger KQ, Saul LK (2009) Distance metric learning for large margin nearest neighbor classification. J Mach Learn Res 10(2)

  13. Bateni P, Goyal R, Masrani V, Wood F, Sigal L (2020) Improved few-shot visual classification. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 14493–14502

  14. Garcia V, Bruna J (2017) Few-shot learning with graph neural networks. arXiv preprint arXiv:1711.04043

  15. Liu G, Zhao L, Li W, Guo D, Fang X (2021) Class-wise metric scaling for improved few-shot classification. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 586–595

  16. Finn C, Abbeel P, Levine S (2017) Modelagnostic meta-learning for fast adaptation of deep networks. In: International Conference on Machine Learning, pp. 1126–1135. PMLR

  17. Yan X, Ye Y, Qiu X, Manic M, Yu H (2019) Cmib: unsupervised image object categorization in multiple visual contexts. IEEE Trans Ind Inform 16(6):3974–3986

    Article  Google Scholar 

  18. Yan X, Mao Y, Ye Y, Yu H, Wang F-Y (2022) Explanation guided cross-modal social image clustering. Inform Sci 593:1–16

    Article  Google Scholar 

  19. Lin Y, Gou Y, Liu X, Bai J, Lv J, Peng X (2022) Dual contrastive prediction for incomplete multi-view representation learning. IEEE Trans Patterna Mach Intell

  20. Buciluǎ C, Caruana R, Niculescu-Mizil A (2006) Model compression. In: Proceedings of the 12th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 535–541

  21. Zhang Y, Xiang T, Hospedales TM, Lu H (2018) Deep mutual learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4320–4328

  22. Yan X, Ye Y, Qiu X, Yu H (2020) Synergetic information bottleneck for joint multi-view and ensemble clustering. Inform Fus 56:15–27

    Article  Google Scholar 

  23. Ravi S, Larochelle H (2016) Optimization as a model for few-shot learning

  24. Nichol A, Achiam J, Schulman J (2018) On first-order meta-learning algorithms. arXivpreprint arXiv:1803.02999

  25. Lee Y, Choi S (2018) Gradient-based metalearning with learned layerwise metric and subspace. In: International Conference on Machine Learning, pp. 2927–2936. PMLR

  26. Grant E, Finn C, Levine S, Darrell T, Griffiths T (2018) Recasting gradient-based metalearning as hierarchical bayes. arXiv preprint arXiv:1801.08930

  27. Zhang R, Che T, Ghahramani Z, Bengio Y, Song Y (2018) Metagan: An adversarial approach to few-shot learning. Adv Neur Inform Proc Syst 31

  28. Rusu AA, Rao D, Sygnowski J, Vinyals O, Pascanu R, Osindero S, Hadsell R (2018) Meta-learning with latent embedding optimization. arXiv preprint arXiv:1807.05960

  29. Jiang X, Havaei M, Varno F, Chartrand G, Chapados N, Matwin S (2018) Learning to learn with conditional class dependencies. In: International Conference on Learning Representations

  30. Widhianingsih TDA, Kang D-K (2021) Augmented domain agreement for adaptable meta-learner on few-shot classification. Appl Intell 1–17

  31. Koch G, Zemel R, Salakhutdinov R et al (2015) Siamese neural networks for one-shot image recognition. In: ICML Deep Learning Workshop, vol. 2, p. 0. Lille

  32. Vinyals O, Blundell C, Lillicrap T, Wierstra D et al (2016) Matching networks for one shot learning. Adv Neur Inform Proc Syst 29

  33. Sung F, Yang Y, Zhang L, Xiang T, Torr PH, Hospedales TM (2018) Learning to compare: Relation network for few-shot learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1199–1208

  34. Oreshkin B, Rodríguez López P, Lacoste A (2018) Tadam: Task dependent adaptive metric for improved few-shot learning. Adv Neur Inform Proc Syst 31

  35. Li W, Xu J, Huo J, Wang L, Gao Y, Luo J (2019) Distribution consistency based covariance metric networks for few-shot learning. Proceedings of the AAAI Conference on Artificial Intelligence 33:8642–8649

    Article  Google Scholar 

  36. Yoon SW, Seo J, Moon J (2019) Tapnet: Neural network augmented with task-adaptive projection for few-shot learning. In: International Conference on Machine Learning, pp. 7115–7123. PMLR

  37. Requeima J, Gordon J, Bronskill J, Nowozin S, Turner RE (2019) Fast and flexible multi-task classification using conditional neural adaptive processes. Adv Neur Inform Proc Syst 32

  38. Zhang C, Cai Y, Lin G, Shen C (2020) Deepemd: Few-shot image classification with differentiable earth mover’s distance and structured classifiers. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 12203–12213

  39. Kim J, Kim T, Kim S, Yoo CD (2019) Edgelabeling graph neural network for few-shot learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 11–20

  40. Liu Y, Lee J, Park M, Kim S, Yang E, Hwang SJ, Yang Y (2018) Learning to propagate labels: Transductive propagation network for few-shot learning. arXiv preprint arXiv:1805.10002

  41. Yang L, Li L, Zhang Z, Zhou X, Zhou E, Liu Y (2020) Dpgn: Distribution propagation graph network for few-shot learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13390–13399

  42. Tang S, Chen D, Bai L, Liu K, Ge Y, Ouyang W (2021) Mutual crf-gnn for few-shot learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 2329–2339

  43. Romero A, Ballas N, Kahou SE, Chassang A, Gatta C, Bengio Y (2014) Fitnets: Hints for thin deep nets. arXiv preprint arXiv:1412.6550

  44. Krizhevsky A, Sutskever I, Hinton GE (2012) Imagenet classification with deep convolutional neural networks. Adv Neur Inform Proc Syst 25

  45. Guo Q, Wang X, Wu Y, Yu Z, Liang D, Hu X, Luo P (2020) Online knowledge distillation via collaborative learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

  46. Liu Y, Cao J, Li B, Yuan C, Hu W, Li Y, Duan Y (2019) Knowledge distillation via instance relationship graph. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

  47. Zhou B, Lapedriza A, Xiao J, Torralba A, Oliva A (2014) Learning deep features for scene recognition using places database. Adv Neur Inform Proc Syst 27

  48. Park W, Kim W, You K, Cho M (2020) Diversified mutual learning for deep metric learning. In: European Conference on Computer Vision, pp. 709–725. Springer

  49. Tian Y, Wang Y, Krishnan D, Tenenbaum JB, Isola P (2020) Rethinking few-shot image classification: A good embedding is all you need? In: European Conference on Computer Vision, pp. 266–282. Springer

  50. Zhou Z, Qiu X, Xie J, Wu J, Zhang C (2021) Binocular mutual learning for improving few-shot classification. In: Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pp. 8402–8411

  51. Wang X, Zhang R, Sun Y, Qi J (2018) Kdgan: Knowledge distillation with generative adversarial networks. Adv Neur Inform Proc Syst 31

  52. Liu Y, Chen K, Liu C, Qin Z, Luo Z, Wang J (2019) Structured knowledge distillation for semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 2604–2613

  53. Minoofam SAH, Bastanfard A, Keyvanpour MR (2021) Trcla: A transfer learning approach to reduce negative transfer for cellular learning automata. IEEE Trans Neur Netw Learn Sys

  54. Gori M, Monfardini G, Scarselli F (2005) A new model for learning in graph domains. In: Proceedings. 2005 IEEE International Joint Conference on Neural Networks, vol. 2, pp. 729–734

  55. Arandjelovic R, Gronat P, Torii A, Pajdla T, Sivic J (2016) Netvlad: Cnn architecture for weakly supervised place recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 5297–5307

  56. Ma J, Xie H, Han G, Chang S-F, Galstyan A, Abd-Almageed W (2021) Partnerassisted learning for few-shot image classification. In: Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pp. 10573–10582

  57. Schroff F, Kalenichenko D, Philbin J (2015) Facenet: A unified embedding for face recognition and clustering. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 815–823 (2015)

  58. Kumar V, Glaude H, de Lichy C, Campbell W (2019) A closer look at feature space data augmentation for few-shot intent classification. arXiv preprint arXiv:1910.04176

  59. Kang D, Kwon H, Min J, Cho M (2021) Relational embedding for few-shot classification. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 8822–8833

  60. Wertheimer D, Tang L, Hariharan B (2021) Few-shot classification with feature map reconstruction networks. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 8012–8021

  61. Xie J, Long F, Lv J, Wang Q, Li P (2022) Joint distribution matters: Deep brownian distance covariance for few-shot classification. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 7972–7981

  62. Tseng H-Y, Lee H-Y, Huang J-B, Yang M-H (2020) Cross-domain few-shot classification via learned feature-wise transformation. arXiv preprint arXiv:2001.08735

  63. Wang Y, Chao W-L, Weinberger KQ, van der Maaten L (2019) Simpleshot: Revisiting nearest-neighbor classification for few-shot learning. arXiv preprint arXiv:1911.04623

  64. Ziko I, Dolz J, Granger E, Ayed IB (2020) Laplacian regularized few-shot learning. In: International Conference on Machine Learning, pp. 11660–11670. PMLR

  65. Hu SX, Moreno PG, Xiao Y, Shen X, Obozinski G, Lawrence ND, Damianou A (2020) Empirical bayes transductive metalearning with synthetic gradients. arXiv preprint arXiv:2004.12696

  66. Zhao Y, Cheung N-M (2023) Fs-ban: Born-again networks for domain generalization few-shot classification. IEEE Trans Image Proc

  67. Ren M, Triantafillou E, Ravi S, Snell J, Swersky K, Tenenbaum JB, Larochelle H, Zemel RS (2018) Meta-learning for semisupervised few-shot classification. arXiv preprint arXiv:1803.00676

  68. Wah C, Branson S, Welinder P, Perona P, Belongie S (2011) The caltech-ucsd birds-200-2011 dataset

  69. Krause J, Stark M, Deng J, Fei-Fei L (2013) 3d object representations for fine-grained categorization. In: Proceedings of the IEEE International Conference on Computer Vision Workshops, pp. 554–561

  70. Zhou B, Lapedriza A, Khosla A, Oliva A, Torralba A (2017) Places: A 10 million image database for scene recognition. IEEE Trans Pattern Anal Mach Intell 40(6):1452–1464

    Article  PubMed  Google Scholar 

  71. Van Horn G, Mac Aodha O, Song Y, Cui Y, Sun C, Shepard A, Adam H, Perona P, Belongie S (2018) The inaturalist species classification and detection dataset. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 8769–8778

  72. Das R, Wang Y-X, Moura JM (2021) On the importance of distractors for fewshot classification. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 9030–9040

  73. Yue Z, Zhang H, Sun Q, Hua X-S (2020) Interventional few-shot learning. Adv Neur Inform Proc Syst 33:2734–2746

    Google Scholar 

  74. Chen W-Y, Liu Y-C, Kira Z, Wang Y-CF, Huang J-B (2019) A closer look at few-shot classification. arXiv preprint arXiv:1904.04232

Download references

Acknowledgements

This work is supported by National Natural Science Foundation of China under grant 61771322, and the Fundamental Research Foundation of Shenzhen under Grant JCYJ20220531100814033.

Author information

Authors and Affiliations

  1. Guangdong Key Laboratory of Intelligent Information Processing and Shenzhen Key Laboratory of Media Security, Shenzhen University, Shenzhen, China

    Min Xiong, Wenming Cao & Zhineng Zhao

  2. State Key Laboratory of Radio Frequency Heterogeneous Integration, Shenzhen University, ShenZhen, China

    Wenming Cao

Authors
  1. Min Xiong
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Wenming Cao
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Zhineng Zhao
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Wenming Cao.

Ethics declarations

Conflicts of interest

We declare that we do not have any commercial or associative interest that represents a conflict of interest in connection with the work submitted.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Xiong, M., Cao, W. & Zhao, Z. Dual-model Collaborative Learning with Knowledge Clustering for Few-shot Image Classification. Multimed Tools Appl 83, 26527–26546 (2024). https://doi.org/10.1007/s11042-023-16551-y

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-023-16551-y

Keywords

Search

Navigation