Graph Embedding Learning for Cross-Modal Information Retrieval

Abstract

The aim of cross-modal retrieval is to learn mappings that project samples from different modalities into a common space where the similarity among instances can be measured. To pursuit common subspace, traditional approaches tend to solve the exact projection matrices while it is unrealistic to fully model multimodal data only by linear projection. In this paper, we propose a novel graph embedding learning framework that directly approximates the projected manifold and utilizes both the labeled information and local geometric structures. It avoids explicit eigenvector decomposition by iterating random walk on graph. Sampling strategies are adopted to generate training pairs to fully explore inter and intra modality among the data cloud. Moreover, graph embedding is learned in a semi-supervised learning manner which helps to discriminate the underlying representation over different classes. Experimental results on Wikipedia datasets show that the proposed framework is effective and outperforms other state-of-the-art methods on cross-modal retrieval.