Abstract
This paper proposes a novel approach for the modeling of semantic correlation between web images and texts. Our approach contains two processes of semantic correlation computing. One is to find the local media objects (LMOs), the components composing text (or image) documents, that match the global semantics of a given image(or text) document based on probabilistic latent semantic analysis (PLSA); The other is to make a direct mapping among LMOs with graph-based learning, with those LMOs achieved based on PLSA as a part of inputs. The two cooperating processes consider both dominant semantics and local subordinate parts of heterogeneous data. Finally, we compute the similarity between the obtained LMOs and a whole document of the same modality and then get the semantic correlation between textual and visual documents. Experimental results demonstrate the effectiveness of the proposed approach.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Wu, X., Qiao, Y., Wang, X., Tang, X.: Cross matching of music and image. In: ACM International Conference on Multimedia, pp. 837–840 (2012)
Jeon, J., Lavrenko, V., Manmatha, R.: Automatic image annotation and retrieval using cross-media relevance models. In: Annual ACM SIGIR Conference, pp. 119–126 (2003)
Wu, L., Jin, R., Jain, A.K.: Tag Completion for Image Retrieval. IEEE Transactions on Pattern Analysis and Machine Intelligence 35, 716–727 (2013)
Wang, M., Ni, B., Hua, X., Chua, T.: Assistive tagging: a survey of multimedia tagging with human-computer joint exploration. ACM Computing Surveys 44, 25–25 (2012)
Rasiwasia, N., Perieira, J.C., Cobiello, E., Doyle, G., Lanckriet, G.R.G., Levy, R., Vasconcelos, N.: A new approach to cross-modal multimedia retrieval. In: MMM, pp. 251–260 (2010)
Zhuang, Y., Yang, Y., Wu, F.: Mining semantic correlation of heterogeneous multimedia data for cross-media retrieval. IEEE Transaction on Multimedia 10, 221–229 (2008)
Jiang, T., Tan, A.: Learning image-text associations. IEEE Transactions on Knowledge and Data Engineering 21, 161–177 (2009)
Monay, F., Perez, D.G.: Modeling semantic aspects for cross-media image indexing. IEEE Transactions on Pattern Analysis and Machine Intelligence 29, 1802–1817 (2007)
Zhou, Y., Liang, M., Du, J.: Study of cross-media topic analysis based on visual topic model. In: 24th Chinese Control and Decision Conference, pp. 3467–3470 (2012)
Zhai, X., Peng, Y., Xiao, J.: Effective heterogeneous similarity measure with nearest neighbors for cross-media retrieval. In: Schoeffmann, K., Merialdo, B., Hauptmann, A.G., Ngo, C.-W., Andreopoulos, Y., Breiteneder, C. (eds.) MMM 2012. LNCS, vol. 7131, pp. 312–322. Springer, Heidelberg (2012)
Zhou, D., Bousquet, O., Lal, T.N., Weston, J., Scholkopf, B.: Learning with local and global consistency. In: NIPS, pp. 237–244 (2003)
Vapnik, V.: Local learning algorithms. Neural Computation 4, 888–900 (1992)
Gao, Y., Wang, M., Zha, Z., Shen, J., Li, X., Wu, X.: Visual-textual joint relevance learning for tag-based social image search. IEEE Transactions on Image Processing 22 (2013)
Hofmann, T.: Unsupervised learning by probabilistic latent semantic analysis. Machine Learning 42, 177–196 (2001)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2015 Springer International Publishing Switzerland
About this paper
Cite this paper
Xue, J., Du, Y., Shui, H. (2015). Semantic Correlation Mining between Images and Texts with Global Semantics and Local Mapping. In: He, X., Luo, S., Tao, D., Xu, C., Yang, J., Hasan, M.A. (eds) MultiMedia Modeling. MMM 2015. Lecture Notes in Computer Science, vol 8936. Springer, Cham. https://doi.org/10.1007/978-3-319-14442-9_48
Download citation
DOI: https://doi.org/10.1007/978-3-319-14442-9_48
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-14441-2
Online ISBN: 978-3-319-14442-9
eBook Packages: Computer ScienceComputer Science (R0)