Skip to main content
SpringerLink
Log in

Geo-location driven image tagging via cross-domain learning

  • Special Issue Paper
  • Published:
Multimedia Systems Aims and scope Submit manuscript

Abstract

With the rapid development of location-based social network, more and more multimedia data are uploaded by users. These data always include large-scale of independent information with both textual and visual contents. To bridge the semantic gap in between, we propose a novel cross-domain learning method for automatic image annotation with geo-location information. First, we propose the topic model-based method for popular concept extraction to adaptively construct cross-domain datasets. Then these concepts are utilized to collect the visual correlation information from Flickr. Finally, we leverage cross-domain learning method for model learning. The comparison experiments on cross-domain datasets are conducted to demonstrate the superiority of the proposed method.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

References

  1. Gao, Y., Tang, J., Hong, R., Dai, Q., Chua, T.-S., Jain, R.: W2go: a travel guidance system by automatic landmark ranking. In: ACM Multimedia, pp. 123–132 (2010)

  2. Ji, R., Duan, L.-Y., Chen, J., Yao, H., Yuan, J., Rui, Y., Gao, W.: Location discriminative vocabulary coding for mobile landmark search. Int. J. Comput. Vis. 96(3), 290–314 (2012)

    Article  MATH  Google Scholar 

  3. Gao, Y., Wang, F., Luan, H., Chua, T.: Brand data gathering from live social media streams. In: ACM Conference on Multimedia Retrieval (2014)

  4. Wang, H., Huang, H., Ding, C.H.Q.: Image annotation using bi-relational graph of images and semantic labels. In: CVPR, pp. 793–800 (2011)

  5. Sang, J., Xu, C., Liu, J.: User-aware image tag refinement via ternary semantic analysis. IEEE Trans. Multimed. 14(3–2), 883–895 (2012)

    Article  Google Scholar 

  6. Gao, Y., Wang, M., Luan, H., Shen, J., Yan, S., Tao, D.: Tag-based social image search with visual-text joint hypergraph learning. In: ACM Multimedia, pp. 1517–1520 (2011)

  7. Belani, A.: Vandalism detection in wikipedia: a bag-of-words classifier approach. In: CoRR, vol. abs/1001.0700 (2010)

  8. Yang, J., Yan, R., Hauptmann, A.: Cross-domain video concept detection using adaptive svms. In: ACM Multimedia (2007)

  9. Jiang, Y., Wang, J., Chang, S., Ngo, C.: Domain adaptive semantic diffusion for large scale context-based video annotation. In: ICCV (2009)

  10. III, H.D.: Frustratingly easy domain adaptation. In: CoRR, vol. abs/0907.1815 (2009)

  11. Wu, P., Dietterich, T.: Improving svm accuracy by training on auxiliary data sources. In: ICML (2004)

  12. Jiang, W., Zavesky, E., Chang, S., Loui, A.: Cross-domain learning methods for high-level visual concept classification. In: ICIP (2008)

  13. Roy, S.D., Mei, T., Zeng, W., Li, S.: Towards cross-domain learning for social video popularity prediction. IEEE Trans. Multimed. 15(6):1 (2013)

  14. Huang, J., Smola, A., Gretton, A., Borgwardt, K., Schölkopf, B.: Correcting sample selection bias by unlabeled data. In: NIPS (2006)

  15. Storkey, A., Sugiyama, M.: Mixture regression for covariate shift. In: NIPS (2006)

  16. Fang, Z., Zhang, Z.M.: Discriminative feature selection for multi-view cross-domain learning. In: CIKM, pp. 1321–1330 (2013)

  17. Chen, L., Duan, L., Tsang, I.W., Xu, D.: Efficient discriminative learning of class hierarchy for many class prediction. In: ACCV, vol. 1, pp. 274–288 (2012)

  18. Bruzzone, L., Marconcini, M.: Domain adaptation problems: a dasvm classification technique and a circular validation strategy. IEEE Trans. Pattern Anal. Mach. Intell. 32(5), 770–787 (2010)

    Article  Google Scholar 

  19. Bruzzone, L., Chi, M., Marconcini, M.: Transductive svms for semisupervised classification of hyperspectral data. In: IGARSS, p. 4 (2005)

  20. Yuan, Y., Wu, F., Shao, J., Zhuang, Y.: Image annotation by semi-supervised cross-domain learning with group sparsity. J. Visual Commun. Image Rep. 24(2), 95–102 (2013)

    Article  Google Scholar 

  21. Si, S., Tao, D., Wang, M., Chan, K.: Social image annotation via cross-domain subspace learning. Multimed. Tools Appl. 56(1), 91–108 (2012)

    Article  Google Scholar 

  22. Federico, L., Nestor, D., Oscar, C.: Smitag: a social network for semantic annotation of medical images. In: CLEI (2012)

  23. Si, S., Tao, D., Chan, K.: Cross-domain web image annotation. In: ICDM Workshops, pp. 184–189 (2009)

  24. Denoyer, L., Gallinari, P.: A ranking based model for automatic image annotation in a social network. In: ICWSM (2010)

  25. Han, Y., Wu, F., Zhuang, Y.: Multi-label image annotation by structural grouping sparsity. In: Social Media Modeling and Computing, pp. 97–118 (2011)

  26. Joshi, D., Luo, J., Yu, J., Lei, P., Gallagher, A.C.: Using geotags to derive rich tag-clouds for image annotation. In: Social Media Modeling and Computing, pp. 239–256 (2011)

  27. Gao, Y., Wang, M., Zha, Z., Shen, J., Li, X., Wu, X.: Visual-textual joint relevance learning for tag-based social image search. IEEE Trans. Image Process. 22(1), 363–373 (2013)

    Article  MathSciNet  Google Scholar 

  28. Blei, D., Ng, A., Jordan, M.: Latent dirichlet allocation. In: NIPS (2001)

  29. Balamurali, A., Mukherjee, S., Malu, A., Bhattacharyya, P.: Leveraging sentiment to compute word similarity. In: CoRR (2012)

  30. Felzenszwalb, P.F., Girshick, R.B., McAllester, D.A., Ramanan, D.: Object detection with discriminatively trained part-based models. IEEE Trans. PAMI 32(9), 1627–1645 (2010)

    Article  Google Scholar 

  31. Viola, P.A., Jones, M.J.: Robust real-time face detection. Int. J. Comput. Vis. 57(2), 137–154 (2004)

    Article  Google Scholar 

  32. Sang, J., Xu, C.: Right buddy makes the difference: an early exploration of social relation analysis in multimedia applications. In: ACM Multimedia, pp. 19–28 (2012)

  33. Ji, R., Yao, H., Liu, W., Sun, X., Tian, Q.: Task-dependent visual-codebook compression. IEEE Trans. Image Process. 21(4), 2282–2293 (2012)

    Article  MathSciNet  Google Scholar 

  34. Yang, J., Yu, K., Gong, Y., Huang, T.: Linear spatial pyramid matching using sparse coding for image classification. In: CVPR (2009)

  35. Sanromà, G., Alquézar, R., Serratosa, F.: A new graph matching method for point-set correspondence using the em algorithm and softassign. Comput. Vis. Image Underst. 116(2), 292–304 (2012)

    Article  Google Scholar 

  36. Pan, S.J., Kwok, J., Yang, Q.: Transfer learning via dimensionality reduction. In: AAAI (2008)

  37. Taylor, M.E., Stone, P.: Cross-domain transfer for reinforcement learning. In: ICML, pp. 879–886 (2007)

  38. Duan, L., Tsang, I., Xu, D.: Domain transfer multiple kernel learning. IEEE Trans. Pattern Anal. Mach. Intell. 34(3), 465–479 (2012)

    Article  Google Scholar 

  39. Lanckriet, G., Cristianini, N., Bartlett, P., Ghaoui, L., Jordan, M.: Learning the kernel matrix with semidefinite programming. J. Mach. Learn. Res. 5, 27–72 (2004)

    MathSciNet  MATH  Google Scholar 

  40. Liu, X., Wang, L., Yin, J., Liu, L.: Incorporation of radius-info can be simple with simplemkl. Neurocomputing 89, 30–38 (2012)

    Article  Google Scholar 

  41. Sonnenburg, S., Rätsch, G., Schäfer, C., Schölkopf, B.: Large scale multiple kernel learning. J. Mach. Learn. Res 7, 1531–1565 (2006)

    MathSciNet  MATH  Google Scholar 

  42. Ji, R., Gao, Y., Zhong, B., Yao, H., Tian, Q.: Mining flickr landmarks by modeling reconstruction sparsity. TOMCCAP 7, 31 (2011)

    Google Scholar 

  43. Dalal, N., Triggs, B.: Histograms of oriented gradients for human detection. In: CVPR, vol. 1, pp. 886–893 (2005)

  44. Mäenpää, T., Ojala, T., Pietikäinen, M., Soriano, M.: Robust texture classification by subsets of local binary patterns. In: ICPR (2000)

  45. Zhang, W., Lu, Y., Xue, X., Fan, J.: Automatic image annotation with weakly labeled dataset. In: ACM Multimedia, pp. 1185–1188 (2011)

  46. Liu, X., Gao, Y., Ji, R., Chang, S., Huang, T. S.: Localizing web videos from heterogeneous images. In: AAAI (late-breaking developments) (2013)

  47. Gao, Y., Dai, Q.: Clip based video summarization and ranking. In: CIVR, pp. 135–140 (2008)

Download references

Acknowledgments

I would like to express my deep gratitude to Prof. Tat-Seng Chua and the NeXT group in National University of Singapore for helpful discussion. This work was supported in part by the National Natural Science Foundation of China (61100124, 21106095, 61170239, and 61202168), the Grant of Elite Scholar Program of Tianjin University, the Grant of Introducing Talents to Tianjin Normal University (5RL123), the Grant of Introduction of One Thousand High-level Talents in Three Years in Tianjin.

Author information

Authors and Affiliations

  1. The department of Electronics Information Engineering, Tianjin University, Tianjin, China

    Weizhi Nie, Anan Liu, Zhongyang Wang & Yuting Su

Authors
  1. Weizhi Nie
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Anan Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Zhongyang Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yuting Su
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Anan Liu.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Nie, W., Liu, A., Wang, Z. et al. Geo-location driven image tagging via cross-domain learning. Multimedia Systems 22, 395–404 (2016). https://doi.org/10.1007/s00530-014-0396-7

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00530-014-0396-7

Keywords

Search

Navigation