Skip to main content
SpringerLink
Log in
Book cover

Social Media Modeling and Computing pp 25–45Cite as

Tag-Based Social Image Search: Toward Relevant and Diverse Results

  • Chapter

Abstract

Recent years have witnessed a great success of social media websites. Tag-based image search is an important approach to access the image content of interest on these websites. However, the existing ranking methods for tag-based image search frequently return results that are irrelevant or lack of diversity. This chapter presents a diverse relevance ranking scheme which simultaneously takes relevance and diversity into account by exploring the content of images and their associated tags. First, it estimates the relevance scores of images with respect to the query term based on both visual information of images and semantic information of associated tags. Then semantic similarities of social images are estimated based on their tags. Based on the relevance scores and the similarities, the ranking list is generated by a greedy ordering algorithm which optimizes Average Diverse Precision (ADP), a novel measure that is extended from the conventional Average Precision (AP). Comprehensive experiments and user studies demonstrate the effectiveness of the approach.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD   109.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

Notes

  1. 1.

    It is worth noting that diversity is not directly related to a user’s search requirements. Therefore, actually the users are asked to take search relevance and comprehensiveness into account. For search comprehensiveness, we asked them to imagine different search intentions when they posed these queries for themselves, and then it is better if the top results in a list cover more possibilities.

References

  1. Agrawal, R., Gollapudi, S., Halverson, A., Leong, S.: Diversifying search results. In: Proceedings of ACM International Conference on Web Search and Data Mining (2009)

    Google Scholar 

  2. Buckley, C., Voorhees, E.M.: Retrieval evaluation with incomplete information. In: SIGIR (2004)

    Google Scholar 

  3. Cai, D., He, X., Li, Z., Ma, W.-Y., Wen, J.-R.: Hierarchical clustering of WWW image search results using visual, textual and link information. In: Proceedings of ACM Multimedia, pp. 952–959 (2004)

    Google Scholar 

  4. Carbonell, J., Goldstein, J.: The use of MMR, diversity-based reranking for reordering documents and producing summaries. In: Proceedings of SIGIR, pp. 335–336 (1998)

    Google Scholar 

  5. Chen, H., Karger, D.R.: Less is more: probabilistic models for retrieving fewer relevant documents. In: Proceedings of the 29th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, p. 436. ACM, New York (2006)

    Google Scholar 

  6. Cilibrasi, R., Vitanyi, P.M.B.: The google similarity distance. IEEE Trans. Knowl. Data Eng. 19, 370–383 (2007)

    Article  Google Scholar 

  7. Clarke, C.L.A., Kolla, M., Cormack, G.V., Vechtomova, O., Ashkan, A., Büttcher, S., MacKinnon, I.: Novelty and diversity in information retrieval evaluation. In: Proceedings of the 31st Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 659–666. ACM, New York (2008)

    Chapter  Google Scholar 

  8. Frey, B.J., Dueck, D.: Clustering by passing messages between data points. Science 315, 972–976 (2007)

    Article  MathSciNet  Google Scholar 

  9. Goffman, W.: A searching procedure for information retrieval. In: Information Storage and Retrieval, vol. 2, pp. 73–78 (1964)

    Google Scholar 

  10. Hsu, W.H., Kennedy, L.S., Chang, S.-F.: Video search reranking via information bottleneck principle. In: Proceedings of ACM Multimedia, pp. 35–44 (2006)

    Google Scholar 

  11. Jaimes, A., Chang, S.-F., Loui, A.C.: Detection of non-identical duplicate consumer photographs. In: Proceedings of ACM Multimedia, pp. 16–20 (2003)

    Google Scholar 

  12. Järvelin, K., Kekäläinen, J.: Cumulated gain-based evaluation of IR techniques. ACM Trans. Inf. Syst. 20(4), 446 (2002)

    Article  Google Scholar 

  13. Jing, F., Wang, C., Yao, Y., Deng, K., Zhang, L., Ma, W.-Y.: IGroup: web image search results clustering. In: Proceedings of ACM Multimedia, pp. 587–596 (2006)

    Google Scholar 

  14. Kennedy, L.S., Chang, S.F., Kozintsev, I.V.: To search or to label? predicting the performance of search-based automatic image classifiers. In: Proceedings of MIR, pp. 249–258 (2006)

    Google Scholar 

  15. Kennedy, L., Slaney, M., Weinberger, K.: Reliable tags using image similarity: mining specificity and expertise from large-scale multimedia databases. In: WSMC ’09: Proceedings of the 1st Workshop on Web-scale Multimedia Corpus, pp. 17–24. ACM, New York (2009)

    Chapter  Google Scholar 

  16. King, B.M., Minium, E.W.: Statistical reasoning in psychology and education. Wiley, New York (2003)

    Google Scholar 

  17. Li, J., Wang, J.: Real-time computerized annotation of pictures. IEEE Trans. Pattern Anal. Mach. Intell. 30(6), 985–1002 (2008)

    Article  Google Scholar 

  18. Li, X.R., Snoek, C.G.M., Worring, M.: Learning tag relevance by neighbor voting for social image retrieval. In: Proceedings of MIR, pp. 180–187 (2008)

    Chapter  Google Scholar 

  19. Liu, D., Wang, M., Yang, L., Hua, X.-S., Zhang, H.-J.: Tag quality improvement for social images. In: Proceedings of ICME, pp. 350–353 (2009)

    Google Scholar 

  20. Nah, F.F.-H.: A study on tolerable waiting time: how long are web users willing to wait. Behav. Inf. Technol. 23(3), 153–163 (2004)

    Article  Google Scholar 

  21. Qi, G.J., Hua, X.S., Rui, Y., Tang, J.H., Zha, Z.J., Zhang, H.J.: A joint appearance-spatial distance for kernel-based image categorization. In: Proceedings of CVPR, pp. 1–8 (2008)

    Google Scholar 

  22. Robertson, S.: The probability ranking principle in IR. J. Doc. 33(294), 294–304 (1977)

    Article  Google Scholar 

  23. Rui, Y., Huang, T.S.: Relevance feedback: a power tool for interactive content-based image retrieval. IEEE Trans. Circuits Syst. Video Technol. 8(5), 644–655 (1999)

    Google Scholar 

  24. Santos, R.L.T., Macdonald, C., Ounis, I.: Exploiting query reformulations for Web search result diversification. In: Proceedings of the 19th International Conference on World Wide Web, pp. 881–890. ACM, New York (2010)

    Chapter  Google Scholar 

  25. Song, K., Tian, Y., Huang, T., Gao, W.: Diversifying the image retrieval results. In: Proceedings of ACM Multimedia, pp. 707–710 (2006)

    Google Scholar 

  26. Srinivasan, S.H., Sawant, N.: Finding near-duplicate images on the web using fingerprints. In: Proceedings of ACM Multimedia, pp. 881–884 (2008)

    Google Scholar 

  27. Sun, A., Bhowmick, S.S.: Image tag clarity: in search of visual-representative tags for social images. In: WSM ’09: Proceedings of the First SIGMM Workshop on Social Media, pp. 19–26. ACM, New York (2009)

    Chapter  Google Scholar 

  28. Van Leuken, R.H., Garcia, L., Olivares, X., Zwol, R.: Visual diversification of image search results. In: Proceedings of WWW, pp. 341–350 (2009)

    Chapter  Google Scholar 

  29. Wang, B., Li, Z., Li, M., Ma, W.-Y.: Large-scale duplicate detection for web image search. In: Proceedings of ICME, pp. 353–356 (2006)

    Google Scholar 

  30. Wang, M., Hua, X.-S., Tang, J., Hong, R.: Beyond distance measurement: constructing neighborhood similarity for video annotation. IEEE Trans. Multimed. 11(3), 465–476 (2009)

    Article  Google Scholar 

  31. Weinberger, K.Q., Slaney, M., Van Zwol, R.: Resolving tag ambiguity. In: MM ’08: Proceeding of the 16th ACM International Conference on Multimedia, pp. 111–120. ACM, New York (2008)

    Chapter  Google Scholar 

  32. Wu, L., Hua, X.-S., Ma, W.-Y., Yu, N., Li, S.: Flickr distance. In: Proceedings of ACM Multimedia, pp. 31–40 (2008)

    Google Scholar 

  33. Yahoo key scientific challenges program. http://research.yahoo.com/ksc/multimedia

  34. Yang, K., Wang, M., Hua, X.-S., Zhang, H.-J.: Social image search with diverse relevance ranking. In: International MultiMedia Modeling Conference (MMM) (2010)

    Google Scholar 

  35. Zhai, C., Cohen, W.W., Lafferty, J.: Beyond independent relevance: methods and evaluation metrics for subtopic retrieval. In: Information Processing and Management, pp. 10–17 (2006)

    Google Scholar 

  36. Zhai, C., Lafferty, J.: A risk minimization framework for information retrieval. Inf. Process. Manag. 31–55 (2006)

    Google Scholar 

  37. Zhao, W.L., Ngo, C.W.: Scale-rotation invariant pattern entropy for keypoint-based near-duplicate detection. IEEE Trans. Image Process. 18(2), 412–423 (2009)

    Article  MathSciNet  Google Scholar 

  38. Zhu, J., Hoi, S.C.H., Lyu, M.R., Yan, S.: Near-duplicate keyframe retrieval by nonrigid image matching. In: Proceedings of ACM Multimedia, pp. 41–50 (2008)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Automation, The University of Science and Technology of China, Hefei, Anhui, 230027, China

    Kuiyuan Yang

  2. AKiiRA Media Systems Inc, Palo Alto, CA, 94301, USA

    Meng Wang

  3. Media Computing Group, Microsoft Research Asia, Beijing, 100080, China

    Xian-Sheng Hua

  4. Microsoft Advanced Technology Center, Beijing, 100080, China

    Hong-Jiang Zhang

Authors
  1. Kuiyuan Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Meng Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Xian-Sheng Hua
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Hong-Jiang Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Kuiyuan Yang .

Editor information

Editors and Affiliations

  1. School of Computer Engineering, Nanyang Technological University, Singapore, 639798, Singapore

    Steven C.H. Hoi

  2. Kodak Research Laboratories, Lake Avenue 1999, Rochester, 14650, New York, USA

    Jiebo Luo

  3. Media Informatics and Multimedia Systems, University of Oldenburg, Escherweg 2, Oldenburg, 26121, Germany

    Susanne Boll

  4. School of Computer Engineering, Nanyang Technological University, Singapore, 639798, Singapore

    Dong Xu

  5. Dept. Computer Science and Engineering, Michigan State University, Engineering Building 3115, East Lansing, 48824, Michigan, USA

    Rong Jin

  6. Dept. Computer Science and Engineering, The Chinese University of Hong Kong, Shatin, Hong Kong/PR China

    Irwin King

Rights and permissions

Reprints and permissions

Copyright information

© 2011 Springer-Verlag London Limited

About this chapter

Cite this chapter

Yang, K., Wang, M., Hua, XS., Zhang, HJ. (2011). Tag-Based Social Image Search: Toward Relevant and Diverse Results. In: Hoi, S., Luo, J., Boll, S., Xu, D., Jin, R., King, I. (eds) Social Media Modeling and Computing. Springer, London. https://doi.org/10.1007/978-0-85729-436-4_2

Download citation

  • DOI: https://doi.org/10.1007/978-0-85729-436-4_2

  • Publisher Name: Springer, London

  • Print ISBN: 978-0-85729-435-7

  • Online ISBN: 978-0-85729-436-4

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation