Skip to main content
Springer Nature Link
Log in

Class-Incremental Generalized Zero-Shot Learning

  • 1227: Content-based Image Retrieval
  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Zero-Shot Learning (ZSL) focuses on transferring knowledge learned from the source domain to the target domain. In the classic setting, test data only come from the target domain. Recently, a more reasonable setting Generalized ZSL (G-ZSL) evaluates the model by recognizing instances from both the source and the target domain. However, G-ZSL still have some unrealistic assumptions. One of such hypotheses is the class-fixed setting: during the training time, G-ZSL algorithms only learn to recognize a fixed class set, and we cannot modify the target model unless retraining. The capacity of learning continuously and efficiently is crucial for learning algorithms in a real scenario. In this paper, we extend G-ZSL to a more realistic setting: instead of supposing the class-fixed training strategy, the incoming data come in the way of class-incremental order. In different learning episodes, disjoint groups of categories are utilized to train the G-ZSL models. As the training process goes on, the source domain is expanding. In such a Class-Incremental Generalized Zero-Shot Learning (CIG-ZSL) setting, learning algorithms are expected to not only transfer the learned knowledge to the target domain but also to remember the previously learned knowledge. We propose a Dual Path Learner (DPL) algorithm to validate the possibility of solving the CIG-ZSL task. Experiments on several benchmarks show that DPL has the capacity of remembering the knowledge learned from previous source instances and be able to migrate all the knowledge to the target domain as the expanding of the source domain.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Algorithm 1
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

References

  1. Akata Z, Perronnin F, Harchaoui Z, Schmid C (2016) Label-embedding for image classification. IEEE Trans Pattern Anal Mach Intell 38(7):1425–1438

    Article  Google Scholar 

  2. Ben-David S, Kushilevitz E, Mansour Y (1997) Online learning versus offline learning. Mach Learn 29(1):45–63

    Article  MATH  Google Scholar 

  3. Bendale A, Boult T (2015) Towards open world recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 1893–1902

  4. Candy PC (1991) Self-Direction for Lifelong Learning. A Comprehensive Guide to Theory and Practice, ERIC

    Google Scholar 

  5. Chao WL, Changpinyo S, Gong B, Sha F (2016) An empirical study and analysis of generalized zero-shot learning for object recognition in the wild. In ECCV

  6. Deng J, Dong W, Socher R, Li LJ, Li K, Fei-Fei L (2009) Imagenet: A large-scale hierarchical image database. In CVPR

  7. Farhadi A, Endres I, Hoiem D, Forsyth D (2009) Describing objects by their attributes. In Computer Vision and Pattern Recognition, 2009. CVPR 2009. IEEE Conference on, pages 1778–1785. IEEE

  8. Felix R, Kumar VBG, Reid I, Carneiro G (2018) Multi-modal cycle-consistent generalized zero-shot learning. In Proceedings of the European Conference on Computer Vision (ECCV), pages 21–37

  9. Field J (2000) Lifelong learning and the new educational order. ERIC

  10. Frome A, Corrado GS, Shlens J, Bengio S, Dean J, Ranzato MA, Mikolov T (2013) DeViSE: A deep visual-semantic embedding model. In NIPS

  11. Fu Y, Hospedales TM, Xiang T, Gong S (2015) Transductive multi-view zero-shot learning. IEEE Trans Pattern Anal Mach Intell 37(11):2332–2345

    Article  Google Scholar 

  12. Fu Y, Sigal L (2016) Semi-supervised vocabulary-informed learning. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 5337–5346

  13. Fu Y, Xiang T, Jiang YG, Xue X, Sigal L, Gong S (2018) Recent advances in zero-shot recognition: Toward data-efficient understanding of visual content

  14. Kankuekul P, Kawewong A, Tangruamsub S, Hasegawa O (2012) Online incremental attribute-based zero-shot learning. In IEEE Conference on Computer Vision and Pattern Recognition

  15. Kingma D, Ba J (2015) Adam: A method for stochastic optimization. In ICLR

  16. Kirkpatrick J, Pascanu R, Rabinowitz N, Veness J, Desjardins G, Rusu AA, Milan K, Quan J, Ramalho T, Grabska-Barwinska A, et al (2017) Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences, page 201611835

  17. Lampert CH, Nickisch H, Harmeling S (2013) Attribute-based classification for zero-shot visual object categorization. IEEE TPAMI

  18. Lampert CH, Nickisch H, Harmeling S (2014) Attribute-based classification for zero-shot visual object categorization. IEEE Trans Pattern Anal Mach Intell 36(3):453–465

    Article  Google Scholar 

  19. Larochelle H, Erhan D, Bengio Y (2008) Zero-data learning of new tasks. In AAAI

  20. Liu S, Long M, Wang J, Jordan M (2018) Generalized zero-shot learning with deep calibration network. In NIPS

  21. McCloskey M, Cohen NJ (1989) Catastrophic interference in connectionist networks: The sequential learning problem. In Psychology of learning and motivation, volume 24, pages 109–165. Elsevier

  22. Mensink T, Verbeek J, Perronnin F, Csurka G (2013) Distance-based image classification: Generalizing to new classes at near-zero cost. In IEEE TPAMI

  23. Mensink T, Verbeek J, Perronnin F, Csurka G (2012) Metric learning for large scale image classification: Generalizing to new classes at near-zero cost. In ECCV

  24. McCloskey M, Cohen NJ (1989) Catastrophic interference in connectionist networks: The sequential learning problem. Psychology of Learning and Motivation

  25. Norouzi M, Mikolov T, Bengio S, Singer Y, Shlens J, Frome A, Corrado GS, Dean J (2013) Zero-shot learning by convex combination of semantic embeddings. arXiv preprint arXiv:1312.5650

  26. Parikh D, Grauman K (2011) Relative attributes. In ICCV

  27. Patterson G, Hays J (2012) Sun attribute database: Discovering, annotating, and recognizing scene attributes. In IEEE Conference on Computer Vision and Pattern Recognition

  28. Pentina A, Lampert CH (2014) A PAC-bayesian bound for lifelong learning. In International Conference on Machine Learning

  29. Rebuffi SA, Kolesnikov A, Sperl G, Lampert CH (2017) icarl: Incremental classifier and representation learning. In CVPR

  30. Rohrbach M, Stark M, Schiele B (2011) Evaluating knowledge transfer and zero-shot learning in a large-scale setting. In Computer Vision and Pattern Recognition (CVPR), 2011 IEEE Conference on, pages 1641–1648. IEEE

  31. Ruvolo P, Eaton E (2013) Ella: An efficient lifelong learning algorithm. In International Conference on Machine Learning, pages 507–515

  32. Schlimmer JC, Fisher D (1986) A case study of incremental concept induction. In AAAI 86:496–501

    Google Scholar 

  33. Socher R, Ganjoo M, Sridhar H, Bastani O, Manning CD, Ng AY (2013) Zero-shot learning through cross-modal transfer. In NIPS

  34. Thrun S, Mitchell TM (1995) Lifelong robot learning. Robotics and Autonomous Systems

  35. Verma VK, Arora G, Mishra A, Rai P (2018) Generalized zero-shot learning via synthesized examples. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR)

  36. Wah C, Branson S, Welinder P, Perona P, Belongie S (2011) The Caltech-UCSD Birds-200-2011 Dataset. Technical Report CNS-TR-2011-001, California Institute of Technology

  37. Xian Y, Lampert CH, Schiele B, Akata Z (2018) Zero-shot learning-a comprehensive evaluation of the good, the bad and the ugly. IEEE TPAMI

  38. Xian Y, Lorenz T, Schiele B, Akata Z (2018) Feature generating networks for zero-shot learning. In Proceedings of the IEEE conference on computer vision and pattern recognition

  39. Yang FSY, Zhang L, Xiang T, Torr PHS, Hospedales TM (2018) Learning to compare: Relation network for few-shot learning

  40. Yu X, Aloimonos Y (2010) Attribute-based transfer learning for object categorization with zero/one training example. In European Conference on Computer Vision

Download references

Author information

Authors and Affiliations

  1. Shanghai YOFO ROBOT Intelligent Technology Co., Ltd, 200433, Shanghai, China

    Zhenfeng Sun

  2. Fudan University, Organization, 200433, Shanghai, China

    Rui Feng & Yanwei Fu

Authors
  1. Zhenfeng Sun
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Rui Feng
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yanwei Fu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Zhenfeng Sun.

Ethics declarations

Competing Interests and conflicts of interests

All the authours come from Fudan University, China. And the conflicts of interest domain would be fudan.edu.cn. There is no fund supporting this research.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Sun, Z., Feng, R. & Fu, Y. Class-Incremental Generalized Zero-Shot Learning. Multimed Tools Appl 82, 38233–38247 (2023). https://doi.org/10.1007/s11042-023-16316-7

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-023-16316-7

Keywords

Search

Navigation