Skip to main content
SpringerLink
Log in

Learning exclusive discriminative semantic information for zero-shot learning

  • Original Article
  • Published:
International Journal of Machine Learning and Cybernetics Aims and scope Submit manuscript

Abstract

Zero-shot learning (ZSL) aims to recognize unseen classes relying on the knowledge transferred from seen categories. This study presents new methods to solve two main challenges in ZSL. First, as human-annotated semantics are not discriminative enough to identify unseen classes, we propose constructing a novel latent semantic space based on the semantic attributes and designing a class-wise classifier with class-specific information maximize the discrimination of the latent semantics. Besides, to alleviate the common space’s semantic overlapping problem, we first propose constructing exclusive latent class prototypes by exclusive lasso (EL). Second, since previous ZSL methods learn visual-semantic projection between visual features and corresponding single class-level semantics directly, i.e., one-vs-all projection, which neglects the interference caused by background and noises in the image, we leverage the simple quadratic regression to soften this hard constraint. The proposed new model also alleviates the inherent domain shift problem by adopting the dual semantic auto-encoder to connect visual space, semantic space, and latent space, respectively. Comprehensive experiments on five benchmark datasets demonstrate the effectiveness of the proposed model.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

References

  1. Akata Z, Perronnin F, Harchaoui Z, Schmid C (2015) Label-embedding for image classification. IEEE Trans Pattern Anal Mach Intell 38(7):1425–1438

    Article  Google Scholar 

  2. Bartels RH, Stewart GW (1972) Solution of the Matrix Equation AX + XB = C [F4]. Commun ACM 15(9):820–826. https://doi.org/10.1145/361573.361582

    Article  MATH  Google Scholar 

  3. Boyd S, Parikh N, Chu E (2011) Distributed optimization and statistical learning via the alternating direction method of multipliers. Now Publishers Inc

  4. Changpinyo S, Chao W.L, Gong B, Sha F (2016) Synthesized classifiers for zero-shot learning. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 5327–5336

  5. Chen L, Zhang H, Xiao J, Liu W, Chang S.F (2018) Zero-shot visual recognition using semantics-preserving adversarial embedding networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1043–1052

  6. Ding Z, Liu H (2019) Marginalized latent semantic encoder for zero-shot learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 6191–6199

  7. Farhadi A, Endres I, Hoiem D, Forsyth D (2009) Describing objects by their attributes. In: 2009 IEEE conference on computer vision and pattern recognition, pp. 1778–1785. IEEE

  8. Frome A, Corrado G, Shlens J, Bengio S, Dean J, Ranzato M, Mikolov T (2013) Devise: A deep visual-semantic embedding model

  9. Fu Y, Hospedales TM, Xiang T, Gong S (2015) Transductive multi-view zero-shot learning. IEEE Trans Pattern Anal Mach Intell 37(11):2332–2345

    Article  Google Scholar 

  10. Guo Y, Din, G, Han J, Yan C, Zhang J, Dai Q (2019) Landmark selection for zero-shot learning. In: IJCAI, pp. 2435–2441

  11. He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 770–778

  12. Jia Z, Zhang Z, Wang L, Shan C, Tan T (2020) Deep unbiased embedding transfer for zero-shot learning. IEEE Trans Image Process 29:1958–1971

    Article  MathSciNet  MATH  Google Scholar 

  13. Jiang H, Wang R, Shan S, Yang Y, Chen X (2017) Learning discriminative latent attributes for zero-shot classification. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 4223–4232

  14. Kodirov E, Xiang T, Gong S (2017) Semantic autoencoder for zero-shot learning. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 3174–3183

  15. Lampert C.H, Nickisch H, Harmeling S (2009) Learning to detect unseen object classes by between-class attribute transfer. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 951–958. IEEE

  16. Li J, Lan X, Long Y, Liu Y, Chen X, Shao L, Zheng N (2020) A joint label space for generalized zero-shot classification. IEEE Trans Image Process 29:5817–5831

    Article  MathSciNet  MATH  Google Scholar 

  17. Liu J, Li X, Yang G (2018) Cross-class sample synthesis for zero-shot learning. In: BMVC, p. 113

  18. Liu L, Zhou T, Long G, Jiang J, Zhang C (2020) Attribute propagation network for graph zero-shot learning. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 34, pp. 4868–4875

  19. Liu Y, Tuytelaars T (2020) A deep multi-modal explanation model for zero-shot learning. IEEE Trans Image Process 29:4788–4803

    Article  MATH  Google Scholar 

  20. Liu Y, Xie D.Y, Gao Q, Han J, Wang S, Gao X (2019) Graph and autoencoder based feature extraction for zero-shot learning. In: IJCAI, pp. 3038–3044

  21. Van der Maaten L, Hinton G (2008) Visualizing data using t-sne. Journal of machine learning research 9(11)

  22. Mikolov T, Sutskever I, Chen K, Corrado G, Dean J (2013) Distributed representations of words and phrases and their compositionality. arXiv preprint arXiv:1310.4546

  23. Ming D, Ding C (2019) Robust flexible feature selection via exclusive l21 regularization. In: Proceedings of the 28th International Joint Conference on Artificial Intelligence, pp. 3158–3164

  24. Norouzi M, Mikolov T, Bengio S, Singer Y, Shlens J, Frome A, Corrado G.S, Dean J (2013) Zero-shot learning by convex combination of semantic embeddings. arXiv preprint arXiv:1312.5650

  25. Pambala A, Dutta T, Biswas S (2020) Generative model with semantic embedding and integrated classifier for generalized zero-shot learning. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1237–1246

  26. Patterson G, Xu C, Su H, Hays J (2014) The sun attribute database: beyond categories for deeper scene understanding. Int J Comput Vis 108(1–2):59–81

    Article  Google Scholar 

  27. Romera-Paredes B, Torr P (2015) An embarrassingly simple approach to zero-shot learning. In: International conference on machine learning, pp. 2152–2161. PMLR (2015)

  28. Socher R, Ganjoo M, Manning C.D, Ng A (2013) Zero-shot learning through cross-modal transfer. In NIPS

  29. Wah C, Branson S, Welinder P, Perona P, Belongie S (2011) The caltech-ucsd birds-200-2011 dataset

  30. Xian Y, Akata Z, Sharma G, Nguyen Q, Hein M, Schiele B (2016) Latent embeddings for zero-shot classification. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 69–77

  31. Xian Y, Lampert CH, Schiele B, Akata Z (2018) Zero-shot learning-a comprehensive evaluation of the good, the bad and the ugly. IEEE Trans Pattern Anal Mach Intell 41(9):2251–2265

    Article  Google Scholar 

  32. Xie Z, Cao W, Ming Z (2021) A further study on biologically inspired feature enhancement in zero-shot learning. Int J Mach Learn Cybernet 12(1):257–269

    Article  Google Scholar 

  33. Zhang C, Wu T, Zhang Y, Zhao B, Wang T, Cui C, Yin Y (2021) Deep semantic-aware network for zero-shot visual urban perception. International Journal of Machine Learning and Cybernetics pp. 1–15

  34. Zhang H, Koniusz P (2018) Zero-shot kernel learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7670–7679

  35. Zhang H, Long Y, Guan Y, Shao L (2019) Triple verification network for generalized zero-shot learning. IEEE Trans Image Process 28(1):506–517

    Article  MathSciNet  Google Scholar 

  36. Zhang H, Long Y, Zhao C (2018) Attribute relaxation from class level to instance level for zero-shot learning. Electron Lett 54(20):1170–1172

    Article  Google Scholar 

  37. Zhang Z, Saligrama V (2016) Zero-shot learning via joint latent similarity embedding. In: proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 6034–6042

  38. Zhang Z, Xie Y, Yang L (2018) Photographic text-to-image synthesis with a hierarchically-nested adversarial network. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 6199–6208

  39. Zhou Y, Jin R, Hoi S.C.H (2010) Exclusive lasso for multi-task feature selection. In: Proceedings of the thirteenth international conference on artificial intelligence and statistics, pp. 988–995. JMLR Workshop and Conference Proceedings

Download references

Author information

Authors and Affiliations

  1. Chongqing Key Laboratory of Image cognition, Chongqing University of Posts and Telecommunications, Chongqing, 400065, China

    Jian-Xun Mi, Zhonghao Zhang & Debao Tai

  2. College of Computer Science and Technology, Chongqing University of Posts and Telecommunications, Chongqing, 400065, China

    Jian-Xun Mi, Zhonghao Zhang, Debao Tai & Li-Fang Zhou

  3. College of Software Engineering, Chongqing University of Posts and Telecommunications, Chongqing, 400065, China

    Li-Fang Zhou

  4. School of Computers and Information, Hefei University of Technology, Hefei, 230009, China

    Wei Jia

Authors
  1. Jian-Xun Mi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Zhonghao Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Debao Tai
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Li-Fang Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Wei Jia
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jian-Xun Mi.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Mi, JX., Zhang, Z., Tai, D. et al. Learning exclusive discriminative semantic information for zero-shot learning. Int. J. Mach. Learn. & Cyber. 14, 761–772 (2023). https://doi.org/10.1007/s13042-022-01661-0

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s13042-022-01661-0

Keywords

Search

Navigation