Cross-Media Feature Learning Framework with Semi-supervised Graph Regularization

Abstract

With the development of multimedia data, cross-media retrieval has become increasingly important. It can provide the retrieval results with various types of media at the same time by submitting a query of any media type. In cross-media retrieval research, feature learning for different media types is a key challenge. In the existing graph-based methods, the similarity matrix denoting the affinities of data is usually constant matrix. Actually, calculating the similarity matrix based on the distances between the instances can more accurately represent the relevance of multimedia data. Furthermore, the dimensions of the original features are usually very high, which affects the computational time of algorithms. To address the above problems, we propose a novel feature learning algorithm for cross-media data, called cross-media feature learning frame-work with semi-supervised graph regularization (FLGR). FLGR calculates the similarity matrix based on the distances between the projected instances, which can not only accurately protect the relevance of multimedia data, but also effectively reduce the computational time of the algorithm. It explores the sparse and semi-supervised regularization for different media types, and integrates them into a unified optimization problem, which boosts the performance of the algorithm. Furthermore, FLGR studies the semantic information of the original data and further improve the retrieval accuracy. Compared with the current state-of-the-art methods on two datasets, i.e., Wikipedia, XMedia, the experimental results show the effectiveness of our proposed approach.