skip to main content
10.1145/3340531.3411883acmconferencesArticle/Chapter ViewAbstractPublication PagescikmConference Proceedingsconference-collections
research-article

More Than One: A Cluster-Prototype Matching Framework for Zero-Shot Learning

Published: 19 October 2020 Publication History

Abstract

Zero-shot learning (ZSL) aims to recognize unseen categories whose data is unavailable during the training stage. Most existing ZSL algorithms focus on learning an embedding space and determine the classes of test samples according to sample-prototype similarities in this space. However, we observe that, in contrast to the single sample-prototype relationship, an ensemble criterion usually benefits the final classification, just as the saying "more than one". Inspired by this, we introduce a novel cluster-prototype matching (CPM) strategy and propose a ZSL framework based on CPM. Firstly, we learn a mapping between the visual space and the semantic space utilizing a well-established ZSL algorithm. Via the learned mapping, all test samples are projected into the embedding space and clustered in this space. Secondly, two CPM methods, soft-CPM and hard-CPM, are proposed to match clusters and class prototypes, along with cluster-prototype similarities calculated. Finally, the label of each sample is determined by the combination of the sample-prototype similarity and the cluster-prototype similarity. We apply our framework to five basic ZSL methods and compare them with several advanced baselines of ZSL. The experimental results demonstrate that the proposed framework can significantly improve the performance of the basic ZSL models and help them achieve or beyond the state-of-the-art.

Supplementary Material

MP4 File (3340531.3411883.mp4)
In this video, we present our paper ?More Than One: A Cluster-Prototype Matching Framework for Zero-Shot Learning?. In order to better introduce our work, we divide the content of this video into four parts. In the first part, we introduce the background of zero-shot learning and point out that the domain shift problem is the focus of our paper. In the second part, we show our cluster-prototype matching (CPM) framework and explain the details of soft-CPM and hard-CPM. The third part is ?Experiment? and we show ZSL and GZSL performance comparison on five datasets. In this paper, we select nine advanced ZSL methods as baselines and five classical methods as basic models. The experimental results show that our CPM framework can significantly improve the performance of the basic model and help them achieve or beyond the state-of-the-art. In the last part, we give our conclusions about the main advantage of CPM and the analysis for experimental results.

References

[1]
Zeynep Akata, Florent Perronnin, Zaid Harchaoui, and Cordelia Schmid. 2015. Label-embedding for image classification. IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 38, 7 (2015), 1425--1438.
[2]
Soravit Changpinyo, Wei-Lun Chao, Boqing Gong, and Fei Sha. 2016. Synthesized classifiers for zero-shot learning. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 5327--5336.
[3]
Long Chen, Hanwang Zhang, Jun Xiao, Wei Liu, and Shih-Fu Chang. 2018. Zero-shot visual recognition using semantics-preserving adversarial embedding networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 1043--1052.
[4]
Ali Farhadi, Ian Endres, Derek Hoiem, and David Forsyth. 2009. Describing objects by their attributes. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. IEEE, 1778--1785.
[5]
Andrea Frome, Greg S Corrado, Jon Shlens, Samy Bengio, Jeff Dean, Marc'Aurelio Ranzato, and Tomas Mikolov. 2013. Devise: A deep visual-semantic embedding model. In Advances in Neural Information Processing Systems. 2121--2129.
[6]
Rui Gao, Xingsong Hou, Jie Qin, Jiaxin Chen, Li Liu, Fan Zhu, Zhao Zhang, and Ling Shao. 2020. Zero-VAE-GAN: Generating Unseen Features for Generalized and Transductive Zero-Shot Learning. IEEE Transactions on Image Processing, Vol. 29 (2020), 3665--3680.
[7]
Zhong Ji, Hai Wang, Yanwei Pang, and Ling Shao. 2020. Dual triplet network for image zero-shot learning. Neurocomputing, Vol. 373 (2020), 90--97.
[8]
Elyor Kodirov, Tao Xiang, Zhenyong Fu, and Shaogang Gong. 2015. Unsupervised domain adaptation for zero-shot learning. In Proceedings of the IEEE International Conference on Computer Vision. 2452--2460.
[9]
Elyor Kodirov, Tao Xiang, and Shaogang Gong. 2017. Semantic autoencoder for zero-shot learning. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 3174--3183.
[10]
Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. 2012. Imagenet classification with deep convolutional neural networks. In Advances in Neural Information Processing Systems. 1097--1105.
[11]
Christoph H Lampert, Hannes Nickisch, and Stefan Harmeling. 2013. Attribute-based classification for zero-shot visual object categorization. IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 36, 3 (2013), 453--465.
[12]
Hugo Larochelle, Dumitru Erhan, and Yoshua Bengio. 2008. Zero-data learning of new tasks. In Proc. AAAI, Vol. 1. 3.
[13]
Jingjing Li, Mengmeng Jing, Ke Lu, Zhengming Ding, Lei Zhu, and Zi Huang. 2019. Leveraging the invariant side of generative zero-shot learning. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 7402--7411.
[14]
Jin Li, Xuguang Lan, Yang Long, Yang Liu, Xingyu Chen, Ling Shao, and Nanning Zheng. 2020. A joint label space for generalized zero-shot classification. IEEE Transactions on Image Processing (2020).
[15]
Yanan Li and Donghui Wang. 2017. Zero-shot learning with generative latent prototype model. arXiv preprint arXiv:1705.09474 (2017).
[16]
Yang Liu, Q Gao, J Han, S Wang, and X Gao. 2019. Graph and autoencoder based feature extraction for zero-shot learning. In Proc. IJCAI. 15--36.
[17]
Laurens van der Maaten and Geoffrey Hinton. 2008. Visualizing data using t-SNE. Journal of Machine Learning Research, Vol. 9, Nov (2008), 2579--2605.
[18]
James MacQueen et al. 1967. Some methods for classification and analysis of multivariate observations. In Proceedings of the fifth Berkeley symposium on mathematical statistics and probability, Vol. 1. Oakland, CA, USA, 281--297.
[19]
Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S Corrado, and Jeff Dean. 2013. Distributed representations of words and phrases and their compositionality. In Advances in Neural Information Processing Systems. 3111--3119.
[20]
Mark Palatucci, Dean Pomerleau, Geoffrey E Hinton, and Tom M Mitchell. 2009. Zero-shot learning with semantic output codes. In Advances in Neural Information Processing Systems. 1410--1418.
[21]
Genevieve Patterson and James Hays. 2012. Sun attribute database: Discovering, annotating, and recognizing scene attributes. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. IEEE, 2751--2758.
[22]
Bernardino Romera-Paredes and Philip Torr. 2015. An embarrassingly simple approach to zero-shot learning. In International Conference on Machine Learning. 2152--2161.
[23]
Edgar Schonfeld, Sayna Ebrahimi, Samarth Sinha, Trevor Darrell, and Zeynep Akata. 2019. Generalized zero-and few-shot learning via aligned variational autoencoders. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 8247--8255.
[24]
Vinay Kumar Verma and Piyush Rai. 2017. A simple exponential family framework for zero-shot learning. In Joint European Conference on Machine Learning and Knowledge Discovery in Databases. Springer, 792--808.
[25]
Nguyen Xuan Vinh, Julien Epps, and James Bailey. 2010. Information theoretic measures for clusterings comparison: Variants, properties, normalization and correction for chance. The Journal of Machine Learning Research, Vol. 11 (2010), 2837--2854.
[26]
Yinduo Wang, Haofeng Zhang, Zheng Zhang, Yang Long, and Ling Shao. 2020. Learning discriminative domain-invariant prototypes for generalized zero shot learning. Knowledge-Based Systems (2020), 105796.
[27]
Peter Welinder, Steve Branson, Takeshi Mita, Catherine Wah, Florian Schroff, Serge Belongie, and Pietro Perona. 2010. Caltech-UCSD birds 200. (2010).
[28]
Yongqin Xian, Christoph H Lampert, Bernt Schiele, and Zeynep Akata. 2018. Zero-shot learning?A comprehensive evaluation of the good, the bad and the ugly. IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 41, 9 (2018), 2251--2265.
[29]
Li Zhang, Tao Xiang, and Shaogang Gong. 2017. Learning a deep embedding model for zero-shot learning. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2021--2030.
[30]
Ziming Zhang and Venkatesh Saligrama. 2015. Zero-shot learning via semantic similarity embedding. In Proceedings of the IEEE International Conference on Computer Vision. 4166--4174.

Cited By

View all
  • (2024)An Adaptive Feature Selection Method for Learning-to-Enumerate ProblemAdvances in Information Retrieval10.1007/978-3-031-56063-7_8(122-136)Online publication date: 23-Mar-2024

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
CIKM '20: Proceedings of the 29th ACM International Conference on Information & Knowledge Management
October 2020
3619 pages
ISBN:9781450368599
DOI:10.1145/3340531
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 19 October 2020

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. cluster-prototype matching
  2. domain shift
  3. image classification
  4. zero-shot learning

Qualifiers

  • Research-article

Funding Sources

  • Beijing Natural Science Foundation
  • National Key Research and Development Program of China
  • Research and Development Program of China Railways Corporation
  • National Natural Science Foundation of China

Conference

CIKM '20
Sponsor:

Acceptance Rates

Overall Acceptance Rate 1,861 of 8,427 submissions, 22%

Upcoming Conference

CIKM '25

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)12
  • Downloads (Last 6 weeks)0
Reflects downloads up to 17 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2024)An Adaptive Feature Selection Method for Learning-to-Enumerate ProblemAdvances in Information Retrieval10.1007/978-3-031-56063-7_8(122-136)Online publication date: 23-Mar-2024

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media