Abstract
With the rapidly increasing popularity of social media websites, large numbers of images with user-annotated tags are uploaded by web users. Developing automatic techniques to retrieval such massive social images attracts much attention of researchers. The method of social image search returns top-k images according to several keywords input by users. However, the returned results by existing methods are usually irrelevant or lack of diversity, which cannot satisfy user’s veritable intention. In this paper, we propose an effective and efficient re-ranking framework for social image search, which can quickly and accurately return ranking results. We not only consider the consistency of visual content of images and semantic interpretations of tags, but also maximize the coverage of the user’s query demand. Specifically, we first build a social relationship graph by exploring the heterogeneous attribute information of social networks. For a given query, to ensure the effectiveness, we execute an efficient keyword search algorithm over the social relationship graph, and obtain top-k relevant candidate results. Moreover, we propose a novel re-ranking optimization strategy to refine the candidate results. Meanwhile, we develop an index to accelerate the optimization process, which ensures the efficiency of our framework. Extensive experimental conducts on real-world datasets demonstrate the effectiveness and efficiency of proposed re-ranking framework.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Wu, Y., Cao, N., Gotz, D., Tan, Y., Keim, D.: A survey on visual analytics of social media data. IEEE Trans. Multimed. 18(11), 2135–2148 (2016)
Chen, L., Xu, D., Tsang, W., Luo, D.: Tag-based web photo retrieval imapproved by batch mode re-tagging. In: CVPR, pp. 3440–3446 (2010)
Liu, D., Wang, M., Yang, L., Hua, X., Zhang, H.: Tag quality improvement for social images. In: ACM Multimedia, pp. 350–353 (2009)
Liu, D., Yan, D., Hua, S., Zhang, H.: Image retagging using collaborative tag propagation. IEEE Trans. Multimed. 13(4), 702–712 (2011)
Zhu, G., Yan, S., Ma, Y.: Image tag refinement towards low-rank, content-tag prior and error sparsity. In: ACM Multimeida, pp. 461–470 (2010)
Yang, K., Hua, X., Wang, M., Zhang, H.: Tag tagging: towards more descriptive keywords of image content. IEEE Trans. Multimed. 13(4), 662–673 (2011)
Gao, Y., Wang, M., Zha, Z., Shen, J., Li, X.: Visual-textual joint relevance learning for tag-based social image search. IEEE Trans. Image Process. 22(1), 363–376 (2013)
Seah, B., Bhowmick, S., Sun, A.: PRISM: concept-preserving social image search results summarization. PVLDB 8(12), 1868–1871 (2015)
Huang, F., Zhang, X., Li, Z., He, Y., Zhao, Z.: Learning social image embedding with deep multimodal attention networks. In: ACM Multimedia, pp. 460–468 (2017)
Dao, M., Minh, P., Kasem, A., Nazmudeen, M.: A context-aware late-fusion approach for disaster image retrieval from social media. In: ICMR, pp. 266–273 (2018)
Chen, Y., Tsai, Y., Li, C.: Query embedding learning for context-based social search. In: CIKM, pp. 2441–2444 (2018)
Wu, B., Jia, J., Yang, Y., Zhao, P., Tian, Q.: Inferring emotional tags from social images with user demographics. IEEE Trans. Multimed. 19(7), 1670–1684 (2017)
Zhang, J., Yang, Y., Tian, Q., Liu, X.: Personalized social image recommendation method based on user-image-tag model. IEEE Trans. Multimed. 19(11), 2439–2449 (2017)
Lu, D., Liu, X., Qian, X.: Tag-based image search by social re-ranking. IEEE Trans. Multimed. 18(8), 1628–1639 (2016)
Tremeau, A., Colantoni, P.: Regions adjacency graph applied to color image segmentation. IEEE Trans. Image Process 9(4), 735–744 (2000)
He, H., Singh, A.: Closure-tree: an index structure for graph queries. In: ICDE, pp. 38–49 (2006)
Zhuang, F., Mei, T., Steven, C., Hua, S.: Modeling social strength in social media community via kernel-based learning. In: ACM Multimedia, pp. 113–122 (2011)
Cortes, C., Mohri, M., Rostamizadeh, A.: Two-stage learning kernel algorithms. In: ICML, pp. 239–246 (2010)
Comanicu, D., Meer, P.: Mean shift: a robust approach toward feature space analysis. IEEE Trans. Pattern Anal. Mach. Intell. 24(5), 603–619 (2002)
Lee, J., Hwang, S.: STRG-Index: spatio-temporal region graph indexing for large video databases. In: SIGMOD, pp: 718–729 (2005)
Yuan, J., Li, J., Zhang B.: Exploiting spatial context constraints for automatic image region annotation. In: ACM Multimedia, pp. 595–604 (2007)
Wang, Y., Yuan, Y., Ma, Y., Wang, G.: Time-dependent graphs: definitions, applications, and algorithms. Data Sci. Eng. 4(4), 352–366 (2019). https://doi.org/10.1007/s41019-019-00105-0
Wang, M., Yang, K., Hua, X., Zhang, H.: Towards relevant and diverse search of social images. IEEE Trans. Multimed. 12(8), 829–842 (2010)
Yang, K., Wang, M., Hua, X.S., Zhang, H.J.: Tag-based social image search: toward relevant and diverse results. In: Hoi, S., Luo, J., Boll, S., Xu, D., Jin, R., King, I. (eds.) Social Media Modeling and Computing, pp. 25–45. Springer, London (2011). https://doi.org/10.1007/978-0-85729-436-4_2
Gao, Y., Wang, M., Luan, H., Shen, C.: Tag-based social image search with visual-text joint hypergraph learning. In: ACM Multimedia, pp. 1517–1520 (2017)
Zhang, J., Yang, Y., Tian, Q., Zhuo, L.: Personalized social image recommendation method based on user-image-tag model. IEEE Trans. Multimed. 19(11), 2439–2449 (2017)
Acknowledgements
Bo Lu is supported by the NSFC (Grant No. 61602085), Ye Yuan is supported by the NSFC (Grant No. 61932004, N181605012), Yurong Cheng is supported by the NSFC (Grant No. 61902023, U1811262) and the China Postdoctoral Science General Program Foundation (No. 2018M631358).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Lu, B., Yuan, Y., Cheng, Y., Wang, G., Duan, X. (2020). An Effective and Efficient Re-ranking Framework for Social Image Search. In: Nah, Y., Cui, B., Lee, SW., Yu, J.X., Moon, YS., Whang, S.E. (eds) Database Systems for Advanced Applications. DASFAA 2020. Lecture Notes in Computer Science(), vol 12114. Springer, Cham. https://doi.org/10.1007/978-3-030-59419-0_22
Download citation
DOI: https://doi.org/10.1007/978-3-030-59419-0_22
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-59418-3
Online ISBN: 978-3-030-59419-0
eBook Packages: Computer ScienceComputer Science (R0)