Abstract:
Cross-modal retrieval has become a hot issue in past years. Many existing works pay attentions on correlation learning to generate a common subspace for cross-modal corre...Show MoreNotes: IEEE Xplore ® Notice to Reader “Adversarial Learning-Based Semantic Correlation Representation for Cross-Modal Retrieval” by Lei Zhu, Jiayu Song, Xiaofeng Zhu, Chengyuan Zhang, Shichao Zhang, and Xinpan Yuan published in IEEE MultiMedia, pp. 79-90, October-December 2020. Digital Object Identifier: 10.1109/MMUL.2020.3015764 The corresponding authors of this article are Chengyuan Zhang (e-mail: cyzhangcse@hnu.edu.cn) and Shichao Zhang (e-mail: zhangsc@csu.edu.cn). We regret any inconvenience this may have caused. Shu-Ching Chen Editor-in-Chief IEEE Multimedia
Metadata
Abstract:
Cross-modal retrieval has become a hot issue in past years. Many existing works pay attentions on correlation learning to generate a common subspace for cross-modal correlation measurement, and others use adversarial learning technique to abate the heterogeneity of multimodal data. However, very few works combine correlation learning and adversarial learning to bridge the intermodal semantic gap and diminish cross-modal heterogeneity. This article proposes a novel cross-modal retrieval method, named Adversarial Learning based Semantic COrrelation Representation (ALSCOR), which is an end-to-end framework to integrate cross-modal representation learning, correlation learning, and adversarial. Canonical correlation analysis model, combined with VisNet and TxtNet, is proposed to capture cross-modal nonlinear correlation. Besides, intramodal classifier and modality classifier are used to learn intramodal discrimination and minimize the intermodal heterogeneity. Comprehensive experiments are conducted on three benchmark datasets. The results demonstrate that the proposed ALSCOR has better performance than the state of the arts.
Notes: IEEE Xplore ® Notice to Reader “Adversarial Learning-Based Semantic Correlation Representation for Cross-Modal Retrieval” by Lei Zhu, Jiayu Song, Xiaofeng Zhu, Chengyuan Zhang, Shichao Zhang, and Xinpan Yuan published in IEEE MultiMedia, pp. 79-90, October-December 2020. Digital Object Identifier: 10.1109/MMUL.2020.3015764 The corresponding authors of this article are Chengyuan Zhang (e-mail: cyzhangcse@hnu.edu.cn) and Shichao Zhang (e-mail: zhangsc@csu.edu.cn). We regret any inconvenience this may have caused. Shu-Ching Chen Editor-in-Chief IEEE Multimedia
Published in: IEEE MultiMedia ( Volume: 27, Issue: 4, 01 Oct.-Dec. 2020)