Skip to main content
SpringerLink
Log in

LGSim: local task-invariant and global task-specific similarity for few-shot classification

  • Original Article
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

Few-shot learning is one of the most challenging problems in computer vision due to the difficulty of sample collection in many real-world applications. It aims at classifying a sample when the number of training samples for each identity is limited. Most of the existing few-shot learning models learn a distance metric with pairwise or triplet constraints. In this paper, we make initial attempts on learning local and global similarities simultaneously to improve the few-shot classification performance in terms of accuracy. In particular, our system differs in two aspects. Firstly, we develop a neural network to learn the pairwise local relationship between each pair of samples in the union set that is composed of support set and query set, which fully utilize the supervision. Secondly, we design a global similarity function from the manifold perspective. The latent assumption is that if the neighbors of one sample are similar to those of another sample, the global similarity between them will be high. Otherwise, the global similarity of the two samples will become very low even if the local similarity between them is high. Meanwhile, we propose a new loss according to the pairwise local loss and task-specific global loss, encouraging the model toward better generalization. Extensive experiments on three popular benchmarks (Omniglot, miniImageNet and tieredImageNet) demonstrate that our simple, yet effective approach can achieve competitive accuracy compared to the state-of-the-art methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

References

  1. Szegedy C, Liu W, Jia Y et al (2015) Going deeper with convolutions. In: IEEE conference on computer vision and pattern recognition, CVPR, Boston, MA, USA, pp 1–9

  2. Silver D, Huang A, Maddison CJ et al (2016) Mastering the game of Go with deep neural networks and tree search. Nature 529:484–489

    Article  Google Scholar 

  3. Krizhevsky A, Sutskever I, Hinton GE (2015) ImageNet classification with deep convolutional neural networks. In: 26th annual conference on neural information processing systems, Lake Tahoe, Nevada, United States, pp 1106–1114

  4. Haijun Z, Yuzhu J, Wang H et al (2018) Sitcom-star-based clothing retrieval for video advertising: a deep learning framework. Neural Comput Appl 31(11):7361–7380

    Google Scholar 

  5. Perez L, Wang J (2017) The effectiveness of data augmentation in image classification using deep learning. http://arxiv.org/abs/1712.04621

  6. Marchesi M (2017) Megapixel size image creation using generative adversarial networks. http://arxiv.org/abs/1706.00082

  7. Lemley J, Bazrafkan S (2017) Peter corcoran, smart augmentation learning an optimal data augmentation strategy. IEEE Access 5:5858–5869

    Article  Google Scholar 

  8. Hinton GE, Srivastava N, Krizhevsky A, Sutskever I, Salakhutdinov R (2012) Improving neural networks by preventing co-adaptation of feature detectors. http://arxiv.org/abs/1207.0580

  9. Srivastava N, Hinton GE, Krizhevsky A, Sutskever I, Salakhutdinov R (2014) Dropout: a simple way to prevent neural networks from overfitting. J Mach Learn Res 15:1929–1958

    MathSciNet  MATH  Google Scholar 

  10. Ioffe S, Szegedy C (2015) Batch normalization: accelerating deep network training by reducing internal covariate shift. In: Proceedings of the 32nd international conference on machine learning, ICML, Lille, France, pp 448–456

  11. Wu Y, He K (2018) Group normalization. In: 15th European conference, Munich, Germany, September 8–14, Proceedings, Part XIII, pp 3–19

  12. Altae-Tran H, Ramsundar B, Pappu AS (2016) Low data drug discovery with one-shot learning. http://arxiv.org/abs/1611.03199

  13. Oreshkin BN, Rodriguez Lopez P, Lacoste A (2018) TADAM: task dependent adaptive metric for improved few-shot learning. In: Annual conference on neural information processing systems, NeurIPS, 3–8 December, Montreal, Canada, pp 719–729

  14. Nichol A, Achiam J, Schulman J (2018) On first-order meta-learning algorithms. http://arxiv.org/abs/1803.02999

  15. Andrychowicz M, Denil M, Gomez Colmenarejo S, Hoffman MW, Pfau D, Schaul T, de Freitas N (2016) Learning to learn by gradient descent by gradient descent. In: Annual conference on neural information processing systems, December 5–10, 2016, Barcelona, Spain, pp 3981–3989

  16. Zhou F, Wu B, Li Z (2018) Deep meta-learning: learning to learn in the concept space. http://arxiv.org/abs/1802.03596

  17. Li Z, Zhou F, Chen F, Li H (2017) Meta-SGD: learning to learn quickly for few shot learning. http://arxiv.org/abs/1707.09835

  18. Mao J, Wei X, Yang Y et al (2015) Learning like a child: fast novel visual concept learning from sentence descriptions of images. In: 27th annual conference on neural information processing systems. In: IEEE international conference on computer vision, ICCV, Santiago, Chile, pp 2533–2541

  19. Chen WY, Liu YC, Kira Z et al (2019) A closer look at few-shot classification. In: 7th international conference on learning representations, ICLR, New Orleans, LA, USA

  20. Jiang X, Havaei M, Varno F et al (2019) Learning to learn with conditional class dependencies. In: 7th international conference on learning representations, ICLR, New Orleans, LA, USA

  21. Lake B, Salakhutdinov R, Gross J et al (2011) One shot learning of simple visual concepts. In: Proceedings of the annual meeting of the cognitive science society, vol 33

  22. Finn C, Abbeel P, Levine S (2017) Model-agnostic meta-learning for fast adaptation of deep networks. In: Proceedings of the 34th international conference on machine learning, ICML, Sydney, NSW, Australia, pp 1126–1135

  23. Lee Y, Choi S (2018), Gradient-based meta-learning with learned layerwise metric and subspace. In: Proceedings of the 35th international conference on machine learning, ICML, Stockholmsmassan, Stockholm, Sweden, pp 2933–2942

  24. Jiang X, Havaei M, Varno F, Chartrand G, Chapados N, Matwin S (2019) Learning to learn with conditional class dependencies. In: 7th international conference on learning representations, ICLR 2019, New Orleans, LA, USA

  25. Snell J, Swersky K, Zemel RS (2017) Prototypical networks for few-shot learning. In: Annual conference on neural information processing systems, Long Beach, CA, USA, pp 4077–4087

  26. Ren M, Triantallou E, Ravi S et al (2018) Meta-learning for semi-supervised few-shot classification. In: 6th international conference on learning representations, ICLR, Vancouver, BC, Canada

  27. Fort S (2018) Gaussian prototypical networks for few-shot learning on omniglot. http://arxiv.org/abs/1708.02735

  28. Wang YX, Girshick RB, Hebert M et al (2018) Low-shot learning from imaginary data. In: IEEE conference on computer vision and pattern recognition, CVPR, Salt Lake City, UT, USA, pp 7278–7286

  29. Gregory K, Richard Z, Ruslan S (2015) Siamese neural networks for one-shot image recognition. In: ICML deep learning workshop

  30. Vinyals O, Blundell C, Lillicrap T et al (2016) Matching networks for one shot learning. In: Annual conference on neural information processing systems, Barcelona, Spain, pp 3630–3638

  31. Sung F, Yang Y, Zhang L et al (2018) Learning to compare: relation network for few-shot learning. In: IEEE conference on computer vision and pattern recognition, CVPR, Salt Lake City, UT, USA, pp 1199–1208

  32. Liu Y, Lee J, Park M et al (2018) Learning to propagate labels: transductive propagation network for few-shot learning. In: 7th international conference on learning representations, ICLR, New Orleans, LA, USA

  33. Zhang H, Koniusz P (2019) Power normalizing second-order similarity network for few-shot learning. In: IEEE winter conference on applications of computer vision, WACV, Waikoloa Village, HI, USA, pp 1185–1193

  34. Li FF, Fergus R, Perona P (2003) A bayesian approach to unsupervised one-shot learning of object categories. In: 9th IEEE international conference on computer vision ICCV, Nice, France, pp 1134–1141

  35. Li FF, Fergus R, Perona P (2006) One-shot learning of object categories. IEEE Trans Pattern Anal Mach Intell 28:594–611

    Article  Google Scholar 

  36. Lake BM, Salakhutdinov R, Tenenbaum JB (2013) One-shot learning by inverting a compositional causal process. In: 27th annual conference on neural information processing systems, Lake Tahoe, Nevada, United States, pp 2526–2534

  37. Hilliard N, Phillips L, Howland S et al (2018) Few-shot learning with metric-agnostic conditional embeddings. http://arxiv.org/abs/1802.04376

  38. Ravi S, Larochelle H (2017) Optimization as a model for few-shot learning. In: 5th international conference on learning representations, ICLR 2017, Toulon, France

  39. Triantallou E, Zemel RS, Urtasun R (2017) Few-shot learning through an information retrieval lens. In: Annual conference on neural information processing systems, Long Beach, CA, pp 2255–2265

  40. Ye M, Guo Y (2018) Deep triplet ranking networks for one-shot recognition. http://arxiv.org/abs/1804.07275

  41. Finn C, Xu K, Levine S (2018), Probabilistic model-agnostic meta-learning. In: Annual conference on neural information processing systems, NeurIPS, Montreal, Canada, pp 9537–9548

  42. Rusu AA, Rao D, Sygnowski J et al (2019), Meta-learning with latent embedding optimization. In: 7th international conference on learning representations, ICLR, New Orleans, LA, USA

  43. Lim JJ, Salakhutdinov R, Torralba A (2011) Transfer learning by borrowing examples for multi-class object detection. In: 25th annual conference on neural information processing systems, Granada, Spain, pp 118–126

  44. Long L, Wang W, Wen J et al (2018) Object-level representation learning for few-shot image classification. http://arxiv.org/abs/1805.10777

  45. Zhang R, Che T, Ghahramani Z et al (2018) MetaGAN: an adversarial approach to few-shot learning. In: 7 Advances in neural information processing systems, NeurIPS, Montreal, Canada, pp 2371–2380

  46. Kingma DP, Ba J (2015) Adam: a method for stochastic optimization. In: 3rd international conference on learning representations, ICLR, San Diego, CA, USA

  47. Abadi M, Barham P, Chen J et al (2016) Tensorflow: a system for large-scale machine learning. In: 12th fUSENIXg symposium on operating systems design and implementation (OSDI), vol 16, pp 265–283

  48. Santoro A, Bartunov S, Botvinick M, Wierstra D, Lillicrap TP (2016) One-shot learning with memory-augmented neural networks. http://arxiv.org/abs/1605.06065

Download references

Acknowledgements

This work was financially supported by The Science and Technology Service Network (STS) Double Innovation project of the Chinese Academy of Sciences, the construction and application of the comprehensive management service platform for urban intelligent business travel (Grant No. KFJ-STS-SCYD-017).

Author information

Authors and Affiliations

  1. High Magnetic Field Laboratory, Chinese Academy of Sciences, Hefei, China

    Wenjing Li, Jun Zhang, Tingting Ren & Fang Li

  2. University of Science and Technology of China, Hefei, China

    Wenjing Li & Zhongcheng Wu

  3. Institute of Technology Innovation, Chinese Academy of Sciences, Hefei, China

    Zhongcheng Wu

Authors
  1. Wenjing Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Zhongcheng Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jun Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Tingting Ren
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Fang Li
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Wenjing Li.

Ethics declarations

Conflict of interest

The authors declared no potential conflict of interest with respect to the research, authorship, and/or publication of this article.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Li, W., Wu, Z., Zhang, J. et al. LGSim: local task-invariant and global task-specific similarity for few-shot classification. Neural Comput & Applic 32, 13065–13076 (2020). https://doi.org/10.1007/s00521-020-04750-9

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-020-04750-9

Keywords

Search

Navigation