Skip to main content
SpringerLink
Log in

A survey of tag-based information retrieval

  • Trends and Surveys
  • Published:
International Journal of Multimedia Information Retrieval Aims and scope Submit manuscript

Abstract

This paper aims to provide a comprehensive survey of tag-based information retrieval that covers three areas: tag-based document retrieval, tag-based image retrieval, and tag-based music information retrieval. First of all, seven representative graphical models associated with tag contents are reviewed and evaluated in terms of effectiveness in achieving their goals. The models are explored in depth based on appropriate plate notations for the tag-based document retrieval. Second, well-established review criteria for two-way classical methods, tag refinement and tag recommendation, are utilized for tag-based image retrieval. In particular, tag refinement methods are analyzed by means of the experimental results measured on different datasets. Last, popular tagging methods in the area of music information retrieval are reviewed for the tag-based music information retrieval. We introduce five criteria: used models, tagging purpose, tagging right, object type, and used dataset, for evaluating tag-based information retrieval methods as a new categorical framework engaging the graphical models as well as the two-way classical methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2

Similar content being viewed by others

References

  1. Cisco (2016) Cisco Visual Networking Index: Forecast and Methodology 2015–2020. http://www.cisco.com/c/en/us/solutions/collateral/service-provider/visual-networking-index-vni/complete-white-paper-c11-481360. Accessed 1 June 2016

  2. Gao Y, Wang M, Luan H, Shen J, Yan S, Tao D (2011) Tag-based social image search with visual-text joint hypergraph learning in Proc. ACM ICM, pp 1517–1520

  3. Gao Y, Wang M, Zha Z-J, Shen J, Li X, Wu X (2013) Visual-textual joint relevance learning for tag-based social image search. IEEE Trans Image Process 22(1):363–376

    Article  MathSciNet  Google Scholar 

  4. Zanardi V, Capra L (2008) Social ranking: uncovering relevant content using tag-based recommender systems. In: Proc. ACM Rec Sys, pp 51–58

  5. Nakamoto R, Nakajima S, Miyazaki J, Uemura S (2007) Tag-based contextual collaborative filtering. In: Proc. 18th IEICE data engineering workshop

  6. Carmagnola F, Cena F, Cortassa O, Gena C, Torre I (2007) Towards a tag-based user model: how can user model benefit from tags? User modeling. Springer, Berlin Heidelberg, pp 445–449

    Google Scholar 

  7. Cai Y, Li Q (2010) Personalized search by tag-based user profile and resource profile in collaborative tagging systems. In: Proc. 19th ACM ICIKM, pp 969–978

  8. Chen L, Xu D, Tsang IW, Luo J (2010) Tag-based web photo retrieval improved by batch mode re-tagging. In: Proc. IEEE CVPR, pp 3440–3446

  9. Mathes A (2004) Folksonomies cooperative classification and communication through shared metadata. UIC Technical Report

  10. Trant J (2009) Studying social tagging and folksonomy: A review and framework J. Digital Information 10-1

  11. Gupta M, Li R, Yin Z, Han J (2010) Survey on social tagging techniques. ACM SIGKDD Explor Newslett 12(1):58–72

    Article  Google Scholar 

  12. Dattolo A, Ferrara F, Tasso C (2010) The role of tags for recommendation: a survey. In: HSI, pp 548–555

  13. Majid A, Khusro S, Rauf A (2011) Semantics in social tagging systems: a review. In: ICCNIT, pp 191–203

  14. Wang M, Ni B, Hua XS, Chua TS (2012) Assistive tagging: a survey of multimedia tagging with human–computer joint exploration. ACM CSUR 44(4):25

  15. Deerwester S, Dumais S, Landauer T, Furnas G, Harshman R (1990) Indexing by latent semantic analysis. J Am Soc Inf Sci 41(6):391407

    Article  Google Scholar 

  16. Hofmann T (1999) Probabilistic latent semantic indexing. In: Proceedings of the 22nd annual international ACM SIGIR, pp 50–57

  17. Blei DM, Andrew YN, Michael IJ (2003) Latent dirichlet allocation. J Mach Learn Res 3:993–1022

    MATH  Google Scholar 

  18. Rosen-Zvi M, Griffiths T, Steyvers M, Smyth P (2012) The author-topic model for authors and documents. In: Proceedings of the 20th conference on uncertainty in artificial intelligence. AUAI Press, pp 487–494

  19. Mimno D, McCallum A (2008) Topic models conditioned on arbitrary features with dirichlet-multinomial regression. In: UAI, pp 411–418

  20. Ramage D, Hall D, Nallapati R, Manning CD (2009) Labeled LDA: a supervised topic model for credit attribution in multi-labeled corpora. In: EMNLP, pp 248–256

  21. Ramage D, Manning CD, Dumais S (2011) Partially labeled topic models for interpretable text mining. In: Proc. ACM SIGKDD, pp 457–465

  22. Li S, Li J, Pan R (2013) Tag-weighted topic model for mining semi-structured documents. In: Proceedings of the 23th international joint conference on artificial intelligence. AAAI Press, pp 2855–2861

  23. Li S, Li J, Huang G, Tan R, Pan R (2015) Tag-weighted topic model for large-scale semi-structured documents (arXiv preprint). arXiv:1507.08396

  24. Zamir AR, Ardeshir S, Shah M (2014) GPS-Tag refinement using random walks with an adaptive damping factor. In: CVPR, pp 4280–4287

  25. Liu Y, Wu F, Zhang Y, Shao J, Zhuang Y (2011) Tag clustering and refinement on semantic unity graph. In: ICDM, pp 417–426

  26. Hou Y, Lin Z (2015) Image tag completion and refinement by subspace clustering and matrix completion (arXiv preprint). arXiv:1506.03475

  27. Cui B, Yao J (2012) Tag refinement strategies for social tagging systems U.S. Patent Application 13/980,573, filed March 31

  28. Qian Z, Zhong P, Wang R (2015) Tag refinement for user-contributed images via graph learning and nonnegative tensor factorization. IEEE Signal Process Lett 22(9):1302–1305

    Article  Google Scholar 

  29. Wang L, Zhou TH, Lee YK, Cheoi KJ, Ryu KH (2015) An efficient refinement algorithm for multi-label image annotation with correlation model. Telecommunication Systems, pp 1–17

  30. Sang J, Liu J, Xu C (2011) Exploiting user information for image tag refinement. In: Proceedings of 19th ACM international conference on multimedia, pp 1129–1132

  31. Hua XS, Liu D, Wang M, Zhang HJ (2010) Image Tag Refinement U.S. Patent Application 12/971,880, filed December 17

  32. Jin Y, Khan L, Wang L, Awad M (2005) Image annotations by combining multiple evidence and wordnet. In: Proceedings of 13th annual ACM international conference on multimedia, pp 706–715

  33. Resnik P (1995) Using information content to evaluate semantic similarity in a taxonomy. In: Proceedings of the 14th international joint conference on artificial intelligence

  34. Jiang J, Conrath D (1997) Semantic similarity based on corpus statistics and lexical taxonomy. In: Proceedings of the international conference on research in computational linguistics

  35. Lin D (1997) Using syntatic dependency as a local context to resolve word sense ambiguity. In: Proceedings of the 35th annual meeting of the association for computational linguistics, pp 64–71

  36. Leacock C (1998) Combining local context and wordnet similarity for word sense identification. In: Fellbaum Christiane (ed) WordNet: a lexical reference system and its application. MIT Press, Cambridge, pp 265–283

    Google Scholar 

  37. Banerjee S, Pedersen T (2003) Extended gloss overlaps as a measure of semantic relatedness. In: Proceedings of eighteenth international joint conference on artificial intelligence, pp 805–810

  38. Jeon J, Lavrenko V, Manmatha R (2003) Automatic image annotation and retrieval using cross-media relevance models. In: Proceedings of 26th annual international ACM SIGIR conference on research and development in information retrieval, pp 119–126

  39. Wang C, Jing F, Zhang L, Zhang HJ (2006) Image annotation refinement using random walk with restarts. In: Proceedings of 14th annual ACM international conference on multimedia, pp 647–650

  40. Pan JY, Yang HJ, Faloutsos C, Duygulu P (2004) Automatic multimedia cross-modal correlation discovery. In: Proceedings of 10th ACM SIGKDD international conference on knowledge discovery and data mining, pp 653–658

  41. Cheng W, Wang X (2011) Image tag refinement using tag semantic and visual similarity. In: ICCSNT 4, pp 2146–2149

  42. Wu L, Hua XS, Yu NH, Ma WY, Li SP (2008) Flickr distance ACM multimedia, pp 31–40

  43. Wang C, Jing F, Zhang L, Zhang HJ (2007) Content-based image annotation refinement. In: CVPR’071-8

  44. Sang J, Xu C, Liu J (2012) User-aware image tag refinement via ternary semantic analysis. IEEE Trans Multimedia 14(3):883–895

    Article  Google Scholar 

  45. Liu J, Zhang Y, Li Z, Lu H (2013) Correlation consistency constrained probabilistic matrix factorization for social tag refinement. Neurocomputing 119:3–9

    Article  Google Scholar 

  46. Liu D, Hua XS, Yang L, Wang M, Zhang HJ (2009) Tag ranking. In: Proceedings of 18th international conference on WWW, pp 351–360

  47. Zhuang J, Hoi JCH (2011) A two-view learning approach for image tag ranking. In: WSDM

  48. Zhu G, Yan S, Ma Y (2010) Image tag refinement towards low-rank, content-tag prior and error sparsity. In: Proceedings of international conference on multimedia, pp 461–470

  49. Li Z, Liu J, Zhu X, Liu T, Lu H (2010) Image annotation using multi-correlation probabilistic matrix factorization. ACM Multimedia, pp 1187–1190

  50. Parzen E (1962) On estimation of a probability density function and mode. Ann Math Stat 33(3):1065–1076

  51. Liu D, Yan S, Hua XS, Zhang HJ (2011) Image retagging using collaborative tag propagation. IEEE Trans Multimedia 13(4):702–712

    Article  Google Scholar 

  52. Zhang S, Tian Q, Hua G, Huang Q, Li S (2009) Descriptive visual words and visual phrases for image applications. In: Proc. ACM Int. Conf, Multimedia, p 7584

  53. Xu H, Wang J, Hua XS, Li S (2009) Tag refinement by regularized LDA. In: Proceedings of 17th ACM international conference on multimedia, pp 573–576

  54. Fu J, Wang J, Rui Y, Wang XJ, Mei T (2015) Image tag refinement with view-dependent concept representations. IEEE Trans Circuits Syst Video Technol 25(8):1409–1422

    Article  Google Scholar 

  55. Tsai D, Jing Y, Liu Y, Rowley HA, Ioffe S, Rehg JM (2011) Large-scale image annotation using visual synset. In: Proc. IEEE ICCV, pp 611–618

  56. Chen L, Xu D, Tsang IW, Luo J (2012) Tag-based image retrieval improved by augmented features and group-based refinement. IEEE Trans Multimedia 14(4):10571067

    Article  Google Scholar 

  57. Wu L, Jin R, Jain AK (2013) Tag completion for image retrieval. IEEE Trans Pattern Anal Mach Intell 35(3):716727

    Google Scholar 

  58. Mishne G (2006) AutoTag: a collaborative approach to automated tag assignment for weblog posts. In: Proceedings of 15th international conference on World Wide Web, pp 953–954

  59. Ames M, Naaman M (2007) Why we tag: Motivations for annotation in mobile and online media. In: Proceedings of the SIGCHI conference on human factors in computing systems, pp 971–980

  60. Sigurbj Orsnsson B, van Zwol R (2008) Flickr tag recommendation based on collective knowledge. In: Proceedings of the 17th international conference WWW, pp 327–336

  61. Hotho A, Jaschke R, Schmitz C, Stumme G (2006) Information retrieval in folksonomies: search and ranking. Lect Notes Comput Sci 4011:411–426

    Article  Google Scholar 

  62. Semeraro G, Lops P, Gemmis MD, Musto C, Narducci F (2012) A folksonomy-based recommender system for personalized access to digital artworks. JOCCH 5(3):11

    Article  Google Scholar 

  63. Chen HM, Chang MH, Chang PC, Tien MC, Hsu WH, Wu JL (2008) SheepDog: group and tag recommendation for Flickr photos by automatic search-based learning. In: Proceedings of the 16th ACM international conference on multimedia, pp 737–740

  64. Lee S, Neve WD, Plataniotis KN, Ro YM (2010) MAP-based image tag recommendation using a visual folksonomy. Pattern Recognit Lett 31(9):976–982

    Article  Google Scholar 

  65. Shen Y, Fan J (2010) Leveraging loosely-tagged images and inter-object correlations for tag recommendation. In: Proceedings of international conference on multimedia, pp 5–14

  66. Lindstaedt S, Mrzinger R, Sorschag R, Pammer V, Thallinger G (2009) Automatic image annotation using visual content and folksonomies. Multimedia Tools Appl 42(1):97–113

    Article  Google Scholar 

  67. Silva A, Martins B (2011) Tag recommendation for geo-referenced photos. In: Proceedings of 3rd ACM SIGSPATIAL international workshop on location-based social networks, pp 57–64

  68. Eom W, Lee S, Neve WD, Ro YM (2011) Improving image tag recommendation using favorite image context. In: ICIP, pp 2445–2448

  69. Anderson A, Ranghunathan K, Vogel A (2008) Tagez: Flickr tag recommendation. Association for the Advancement of Artificial Intelligence

  70. Takashita T, Itokawa T, Kitasuka T, Aritsugi M (2010) Tag recommendation for Flickr using web browsing behavior. In: ICCSA, pp 412–421

  71. Xu Z, Fu Y, Mao J, Su D (2006) Towards the semantic web: collaborative tag suggestions. In: Collaborative web tagging workshop at WWW2006

  72. Kim JW, Candan KS, Tatemura J (2010) Organization and tagging of blog and news entries based on content reuse. J Signal Process Syst 58:407–421

    Article  Google Scholar 

  73. Weinberger KQ, Slaney M, Van Zwol R (2008) Resolving tag ambiguity In: MM, pp 111–120

  74. Nguyen QVH, Do ST, Nguyen TT, Aberer K (2015) Tag-based paper retrieval: minimizing user effort with diversity awareness. In: Database systems for advanced applications. Springer International Publishing, pp 510–528

  75. Cantador I, Konstas I, Jose JM (2011) Categorizing social tags to improve folksonomy-based recommendations. Web semantics: science, services and agents on the World Wide Web 9(1):1–15

    Article  Google Scholar 

  76. Wang YS (2014) Image tag recommendation algorithm using tensor factorization. J Multimedia 9(3):416–422

    Article  Google Scholar 

  77. Fang X, Pan R, Cao G, He X, Dai W (2015) personalized tag recommendation through nonlinear tensor factorization using Gaussian kernel. In: AAAI, pp 439–445

  78. McParlane PJ, Moshfeghi J, Jose JM (2013) On contextual photo tag recommendation. In: Proceedings of the 36th international ACM SIGIR conference on research and development in information retrieval, pp 965–968

  79. Turnbull D, Barrington L, Lanckriet G (2008) Five approaches to collecting tags for music. In: ISMIR, vol 8, pp 225–230

  80. Lamere P (2008) Social tagging and music information retrieval. J New Music Res 37(2):101–114

    Article  Google Scholar 

  81. Marlow C, Naaman M, Boyd D, Davis M (2006) Tagging paper, taxonomy, Flickr, academic article, to read. In: Proceedings of ACM hypertext

  82. Ames M, Naaman M (2007) Why we tag: motivations for annotation in mobile and online media. In: CHI 07: proceedings of the SIGCHI conference on Human factors in computing systems, pp 971–980

  83. Nanopoulos A, Rafailidis D, Symeonidis P, Manolopoulos Y (2010) Musicbox: Personalized music recommendation based on cubic analysis of social tags. IEEE Trans Audio Speech Language Process 18(2):407–412

    Article  Google Scholar 

  84. Lathauwer L, Moor B, Vandewalle J (2000) A multilinear singular value decomposition. SIAM J Matrix Anal Appl 21(4):12531278

    MathSciNet  MATH  Google Scholar 

  85. Hariri N, Mobasher B, Burke R (2012) Using social tags to infer context in hybrid music recommendation. In: Proceedings of 12th international workshop on web information and data management, pp 41–48

  86. Nanopoulos A, Karydis I (2011) Know thy neighbor: combining audio features and social tags for effective music similarity. In: ICASSP, pp 165–168

  87. Logan B, Salomon A (2001) A music similarity function based on signal analysis. In: ICME

  88. Yang YH, Bogdanov D, Herrera P, Sordo M (2012) Music retagging using label propagation and robust principal component analysis. In: Proceedings of 21st international conference companion on WWW, pp 869–876

  89. Zhou D, Bousquet O, Lal TN, Weston J, Scholkopf B (2004) Learning with local and global consistency. In: NIPS, pp 321–328

  90. Lin Z, Ganesh A, Wright J, Wu L, Chen M, Ma Y (2009) Fast convex optimization algorithms for exact recovery of a corrupted low-rank matrix Technical Report UILU-ENG-09-2214

  91. Song Y, Dixon S, Pearce M, Fazekas G (2013) Using tags to select stimuli in the study of music and emotion. In: Proc. ICME3

  92. Font F, Serra X (2012) Analysis of the folksonomy of freesound. In: Proc. 2nd comp music workshop

  93. Chen L, Wright P, Nejdl W (2009) Improving music genre classification using collaborative tagging data. In: Proceedings of second ACM international conference on web search and data mining, pp 84–93

  94. Font F, Serra J, Serra X (2014) Audio clip classification using social tags and the effect of tag expansion. In: Audio Engineering Society Conference: 53rd international conference: semantic audio. Audio Engineering Society

  95. Levy M, Sandler M (2007) A semantic space for music derived from social tags. In: Proc. ISMIR

  96. Laurier C, Sordo M, Serra J, Herrera P (2009) Music mood representations from social tags. In: Proc. ISMIR, pp 381–386

  97. Saari P, Eerola T (2014) Semantic computing of moods based on tags in social media of music. IEEE KDE 26(10):2548–2560

    Google Scholar 

  98. Russell JA (1980) A circumplex model of affect. J Person Soc Psychol 39(6):1161–1178

    Article  Google Scholar 

  99. Ekman P (1992) An argument for basic emotions. Cogn Emotion 6:169–200

    Article  Google Scholar 

  100. Ballan L, Bertini M, Uricchio T, Bimbo AD (2013) Social media annotation. In: 11th international workshop on CBMI, pp 229–235

  101. Miller GA, Claudia L, Randee T, Ross TB (1993) A semantic concordance. In: Proceedings of the workshop on human language technology, pp 303–308

  102. Bentivogli L, Pamela F, Bernardo M, Emanuele P (2004) Revising the wordnet domains hierarchy: semantics, coverage and balancing. In: Proceedings of the workshop on multilingual linguistic resources, pp 101–108

  103. Golder SA, Bernardo AH (2006) Usage patterns of col-laborative tagging systems. J Inf Sci 32(2):198–208

  104. Halpin H, Valentin R, Hana S (2007) The complex dynamics of collaborative tagging. In: Proceedings of the 16th international conference on World Wide Web, pp 211–220

  105. You D, Antani S, Demner-Fushman D, Thoma GR. (2014) Does figure-text improve biomedical article retrieval? A pilot study. In: 2014 IEEE 27th international symposium on CBMS, pp 471–472

  106. Simpson M, You D, Rahman MM, Xue Z, Demner-Fushman D, Antani S, Thoma G (2015) Literature-based biomedical image classification and retrieval. Comput Med Imaging Graphics 39:3–13

    Article  Google Scholar 

  107. Wu H, Bowers DM, Huynh TT, Souvenir R (2013) Biomedical video denoising using supervised manifold learning. In: IEEE 10th international symposium on ISBI, pp 1244–1247

  108. Savaris A, Hrder T, Wangenheim A (2014) DCMDSM: a DICOM decomposed storage model. J Am Med Inf Assoc 21(5):917–924

    Article  Google Scholar 

  109. Nemiroff RJ, Bonnell JT (1995) Astronomy picture of the day. Bull Am Astron Soc, 1291. http://antwrp.gsfc.nasa.gov/apod/astropix.html

  110. Kipp M (2011) Tagging of biomedical articles on CiteULike: a comparison of user, author and professional indexing. Knowl Org 38:245–261

    Google Scholar 

  111. Lee S, Yanjun Z, Mohamed Eid M M, Maria V, Semra K, Saeid B (2015) Domain specific information retrieval and text mining in medical document. In: Proceedings of the 6th ACM BCB, pp 67–76

  112. TAC (2016) Proceedings of text analysis conference NIST. https://tac.nist.gov

  113. Turnbull D, Barrington L, Torres D, Lanckriet G (2007) Towards musical query by semantic description using the CAL500 data set. ACM SIGIR, pp 439–446

  114. Knees P, Pampalk E, Widmer G (2004) Artist classification with web-based data. In: Proceeding of ISMIR, pp 517–524

Download references

Author information

Author notes

  1. Sanghoon Lee

    Present address: Department of Neurology, Emory University School of Medicine, Atlanta, GA, USA

Authors and Affiliations

  1. Department of Computer Science, Georgia State University, Atlanta, GA, USA

    Sanghoon Lee, Mohamed Masoud, Janani Balaji, Saeid Belkasim & Rajshekhar Sunderraman

  2. Department of Computer Science, University of Suwon, Hwaseong-si, Gyeonggi-do, South Korea

    Seung-Jin Moon

Authors
  1. Sanghoon Lee
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Mohamed Masoud
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Janani Balaji
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Saeid Belkasim
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Rajshekhar Sunderraman
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Seung-Jin Moon
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Sanghoon Lee.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Lee, S., Masoud, M., Balaji, J. et al. A survey of tag-based information retrieval. Int J Multimed Info Retr 6, 99–113 (2017). https://doi.org/10.1007/s13735-016-0115-6

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s13735-016-0115-6

Keywords

Search

Navigation