Loading [MathJax]/extensions/MathMenu.js
Adversarial Learning-Based Semantic Correlation Representation for Cross-Modal Retrieval | IEEE Journals & Magazine | IEEE Xplore

Adversarial Learning-Based Semantic Correlation Representation for Cross-Modal Retrieval


Abstract:

Cross-modal retrieval has become a hot issue in past years. Many existing works pay attentions on correlation learning to generate a common subspace for cross-modal corre...Show More
Notes: IEEE Xplore ® Notice to Reader “Adversarial Learning-Based Semantic Correlation Representation for Cross-Modal Retrieval” by Lei Zhu, Jiayu Song, Xiaofeng Zhu, Chengyuan Zhang, Shichao Zhang, and Xinpan Yuan published in IEEE MultiMedia, pp. 79-90, October-December 2020. Digital Object Identifier: 10.1109/MMUL.2020.3015764 The corresponding authors of this article are Chengyuan Zhang (e-mail: cyzhangcse@hnu.edu.cn) and Shichao Zhang (e-mail: zhangsc@csu.edu.cn). We regret any inconvenience this may have caused. Shu-Ching Chen Editor-in-Chief IEEE Multimedia

Abstract:

Cross-modal retrieval has become a hot issue in past years. Many existing works pay attentions on correlation learning to generate a common subspace for cross-modal correlation measurement, and others use adversarial learning technique to abate the heterogeneity of multimodal data. However, very few works combine correlation learning and adversarial learning to bridge the intermodal semantic gap and diminish cross-modal heterogeneity. This article proposes a novel cross-modal retrieval method, named Adversarial Learning based Semantic COrrelation Representation (ALSCOR), which is an end-to-end framework to integrate cross-modal representation learning, correlation learning, and adversarial. Canonical correlation analysis model, combined with VisNet and TxtNet, is proposed to capture cross-modal nonlinear correlation. Besides, intramodal classifier and modality classifier are used to learn intramodal discrimination and minimize the intermodal heterogeneity. Comprehensive experiments are conducted on three benchmark datasets. The results demonstrate that the proposed ALSCOR has better performance than the state of the arts.
Notes: IEEE Xplore ® Notice to Reader “Adversarial Learning-Based Semantic Correlation Representation for Cross-Modal Retrieval” by Lei Zhu, Jiayu Song, Xiaofeng Zhu, Chengyuan Zhang, Shichao Zhang, and Xinpan Yuan published in IEEE MultiMedia, pp. 79-90, October-December 2020. Digital Object Identifier: 10.1109/MMUL.2020.3015764 The corresponding authors of this article are Chengyuan Zhang (e-mail: cyzhangcse@hnu.edu.cn) and Shichao Zhang (e-mail: zhangsc@csu.edu.cn). We regret any inconvenience this may have caused. Shu-Ching Chen Editor-in-Chief IEEE Multimedia
Published in: IEEE MultiMedia ( Volume: 27, Issue: 4, 01 Oct.-Dec. 2020)
Page(s): 79 - 90
Date of Publication: 11 August 2020

ISSN Information:

Funding Agency:


References

References is not available for this document.