Stack-VAE Network for Zero-Shot Learning

Abstract

Zero-shot learning aims to transfer knowledge from the seen classes to unseen ones through some high-level semantics (e.g., per-class attributes), enabling the learning model to recognize novel classes without retraining. Among them, the generative methods adopt the scheme that synthesizes samples for the unseen classes, thereby converting the task into a standard classification problem. However, most existing work inevitably suffers from the domain shift problem when only the seen classes are used for supervision. Furthermore, they can not fully leverage the semantic information in data synthesis due to the limited expressiveness of the generator. In this paper, we develop a novel network, named stack-VAE, to alleviate the above problems. The proposal mainly consists of a generative module and a feature core agent. Specifically, we design the generator based on hierarchical VAE, which exploits multi-layer Gaussian distribution to improve the expressiveness, thereby better mimicking the real data distribution of the unseen classes. Besides, we propose a feature core agent based objective, which is beneficial to mitigate seen class bias by enforcing the inter-class separability and reducing the intra-class scatter. Experimental results conducted on three widely used datasets, i.e., AWA2, SUN, CUB, show that the proposed network outperforms the baselines and achieves a new state-of-the-art.

This work was supported in part by the National Natural Science Foundation of China under Grant 62172109 and Grant 62072118, in part by the Natural Science Foundation of Guangdong Province under Grant 2020A1515011361, and in part by the High-level Talents Programme of Guangdong Province under Grant 2017GC010556.