skip to main content
10.1145/1873951.1873984acmconferencesArticle/Chapter ViewAbstractPublication PagesmmConference Proceedingsconference-collections
research-article

iLike: integrating visual and textual features for vertical search

Published: 25 October 2010 Publication History

Abstract

Content-based image search on the Internet is a challenging problem, mostly due to the semantic gap between low-level visual features and high-level content, as well as the excessive computation brought by huge amount of images and high dimensional features. In this paper, we present iLike, a new approach to truly combine textual features from web pages, and visual features from image content for better image search in a vertical search engine. We tackle the first problem by trying to capture the meaning of each text term in the visual feature space, and re-weight visual features according to their significance to the query content. Our experimental results in product search for apparels and accessories demonstrate the effectiveness of iLike and its capability of bridging semantic gaps between visual features and abstract concepts.

References

[1]
Y. A. Aslandogan, C. Thier, C. T. Yu, J. Zou, and N. Rishe. Using semantic contents and wordnet in image retrieval. In SIGIR '97: Proceedings of the 20th annual international ACM SIGIR conference on Research and development in information retrieval, pages 286--295, 1997.
[2]
D. Cai, X. He, Z. Li, W.-Y. Ma, and J.-R. Wen. Hierarchical clustering of www image search results using visual, textual and link information. In MULTIMEDIA '04: Proceedings of the 12th annual ACM international conference on Multimedia, pages 952--959, 2004.
[3]
Z. Chen, L. Wenyin, C. Hu, M. Li, and H.-J. Zhang. ifind: a web image search engine. In SIGIR '01: Proceedings of the 24th annual international ACM SIGIR conference on Research and development in information retrieval, page 450, 2001.
[4]
W. J. Conover. Practical Nonparametric Statistics. John Wiley & Sons, December 1998.
[5]
J. Cui, F. Wen, and X. Tang. Intentsearch: interactive on-line image search re-ranking. In MM '08: Proceeding of the 16th ACM international conference on Multimedia, pages 997--998, 2008.
[6]
J. Cui, F. Wen, and X. Tang. Real time google and live image search re-ranking. In MM '08: Proceeding of the 16th ACM international conference on Multimedia, pages 729--732, 2008.
[7]
J. Cui, F. Wen, R. Xiao, Y. Tian, and X. Tang. Easyalbum: an interactive photo annotation system based on face clustering and re-ranking. In CHI '07: Proceedings of the SIGCHI conference on Human factors in computing systems, pages 367--376, 2007.
[8]
R. Datta, D. Joshi, J. Li, and J. Z. Wang. Image retrieval: Ideas, influences, and trends of the new age. ACM Comput. Surv., 40(2):1--60, 2008.
[9]
T. Deselaers, D. Keysers, and H. Ney. Features for image retrieval -- a quantitative comparison. In DAGM 2004, 2004.
[10]
J.-X. Dua, X.-F. Wang, and G.-J. Zhang. Leaf shape based plant species recognition. Applied Mathematics and Computation, 185(2):883--893, February 2007.
[11]
S. A. Dudani, K. J. Breeding, and R. B. McGhee. Aircraft identification by moment invariants. IEEE Trans. Computers, 26(1):39--46, 1977.
[12]
C. Frankel, M. J. Swain, and V. Athitsos. Webseer: An image search engine for the world wide web. Technical report, Chicago, IL, USA, 1996.
[13]
R. M. Haralick, K. Shanmugam, and I. Dinstein. Textural features for image classification. IEEE Transactions on Systems, Man and Cybernetics, 3(6):610--621, 1973.
[14]
J. Jeon, V. Lavrenko, and R. Manmatha. Automatic image annotation and retrieval using cross-media relevance models. In ACM SIGIR conference on Research and development in informaion retrieval, 2003.
[15]
F. Jing, C. Wang, Y. Yao, K. Deng, L. Zhang, and W.-Y. Ma. Igroup: web image search results clustering. In ACM international conference on Multimedia, 2006.
[16]
P. Kakumanu, S. Makrogiannis, and N. Bourbakis. A survey of skin-color modeling and detection methods. Pattern Recogn., 40(3):1106--1122, 2007.
[17]
I. Kompatsiaris, E. Triantafyllou, and M. Strintzis. A world wide web region-based image search engine. Image Analysis and Processing, International Conference on, 0:0392, 2001.
[18]
P. Kovesi. Image features from phase congruency. Journal of Computer Vision Research, 1(3), 1999.
[19]
S. Lawrence. Context in web search. IEEE Data Engineering Bulletin, 23:25--32, 2000.
[20]
R. Lempel and A. Soffer. Picashow: pictorial authority search by hyperlinks on the web. In WWW '01: Proceedings of the 10th international conference on World Wide Web, pages 438--448, 2001.
[21]
M. S. Lew, N. Sebe, C. Djeraba, and R. Jain. Content-based multimedia information retrieval: State of the art and challenges. ACM Trans. Multimedia Comput. Commun. Appl., 2(1):1--19, 2006.
[22]
X. Li, L. Chen, L. Zhang, F. Lin, and W.-Y. Ma. Image annotation by large-scale content-based image retrieval. In ACM international conference on Multimedia, 2006.
[23]
H. Lieberman, E. Rozenweig, and P. Singh. Aria: An agent for annotating and retrieving images. Computer, 34(7):57--62, 2001.
[24]
B. Luo, X. Wang, and X. Tang. A world wide web based image search engine using text and image content features. In IS&T/SPIE Electronic Imaging, Internet Imaging IV, 2003.
[25]
W.-Y. Ma and H.-J. Zhang. Content-based image indexing and retrieval. Handbook of multimedia computing, pages 227--253, 1998.
[26]
C. D. Manning, P. Raghavan, and H. Schütze. Introduction to information retrieval. Cambridge Univ. Press, New York, NY, 2008.
[27]
S. Mukherjea, K. Hirata, and Y. Hara. Amore: A world wide web image retrieval engine. World Wide Web, 2(3):115--132, 1999.
[28]
S. Sclaroff, L. Taycher, and M. L. Cascia. Imagerover: A content-based image browser for the world wide web. In Workshop on Content-Based Access of Image and Video Libraries (CBAIVL), 1997.
[29]
H. T. Shen, B. C. Ooi, and K.-L. Tan. Giving meanings to www images. In ACM international conference on Multimedia, 2000.
[30]
A. W. M. Smeulders, M. Worring, S. Santini, A. Gupta, and R. Jain. Content-based image retrieval at the end of the early years. IEEE Trans. Pattern Anal. Mach. Intell., 22(12):1349--1380, 2000.
[31]
H. Tamura, S. Mori, and T. Yamawaki. Textural features corresponding to visual perception. IEEE Trans. Systems, Man, and Cybernetics, 8(6), 1978.
[32]
V. S. Tseng, J.-H. Su, B.-W. Wang, and Y.-M. Lin. Web image annotation by fusing visual features and textual information. In ACM symposium on Applied computing (SAC), 2007.
[33]
A. Vijay and M. Bhattacharya. Content-based medical image retrieval using the generic fourier descriptor with brightness. Machine Vision, International Conference on, 0:330--332, 2009.
[34]
L. von Ahn and L. Dabbish. Labeling images with a computer game. In SIGCHI conference on Human factors in computing systems (CHI), 2004.
[35]
S. Wang, F. Jing, J. He, Q. Du, and L. Zhang. Igroup: presenting web image search results in semantic clusters. In SIGCHI conference on Human factors in computing systems (CHI), 2007.
[36]
Z. Wang, Z. Chi, and D. Feng. Fuzzy integral for leaf image retrieval. In Proceedings of the 2002 IEEE International Conference on Fuzzy Systems, 2002.
[37]
K.-P. Yee, K. Swearingen, K. Li, and M. Hearst. Faceted metadata for image search and browsing. In SIGCHI conference on Human factors in computing systems (CHI), 2003.
[38]
L. Zhang, L. Chen, F. Jing, K. Deng, and W.-Y. Ma. Enjoyphoto: a vertical image search engine for enjoying high-quality photos. In ACM international conference on Multimedia, 2006.
[39]
L. Zhang, Y. Hu, M. Li, W. Ma, and H. Zhang. Efficient propagation for face annotation in family albums. In ACM international conference on Multimedia, 2004.
[40]
X. Zhou, M. Wang, Q. Zhang, J. Zhang, and B. Shi. Automatic image annotation by an iterative approach: incorporating keyword correlations and region matching. In ACM international conference on Image and video retrieval (CIVR), 2007.

Cited By

View all
  • (2018)Extracting semantic knowledge from web context for multimedia IRMultimedia Tools and Applications10.1007/s11042-017-4997-y77:11(13853-13889)Online publication date: 1-Jun-2018
  • (2017)Learning Social Circles in Ego-Networks Based on Multi-View Network StructureIEEE Transactions on Knowledge and Data Engineering10.1109/TKDE.2017.268538529:8(1681-1694)Online publication date: 1-Aug-2017
  • (2017)ACSIR: ANOVA Cosine Similarity Image Recommendation in vertical searchInternational Journal of Multimedia Information Retrieval10.1007/s13735-017-0124-06:2(143-154)Online publication date: 31-Mar-2017
  • Show More Cited By

Index Terms

  1. iLike: integrating visual and textual features for vertical search

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Conferences
    MM '10: Proceedings of the 18th ACM international conference on Multimedia
    October 2010
    1836 pages
    ISBN:9781605589336
    DOI:10.1145/1873951
    Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

    Sponsors

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 25 October 2010

    Permissions

    Request permissions for this article.

    Check for updates

    Author Tags

    1. CBIR
    2. specialized search
    3. vertical search engine

    Qualifiers

    • Research-article

    Conference

    MM '10
    Sponsor:
    MM '10: ACM Multimedia Conference
    October 25 - 29, 2010
    Firenze, Italy

    Acceptance Rates

    Overall Acceptance Rate 2,145 of 8,556 submissions, 25%

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)2
    • Downloads (Last 6 weeks)1
    Reflects downloads up to 15 Feb 2025

    Other Metrics

    Citations

    Cited By

    View all
    • (2018)Extracting semantic knowledge from web context for multimedia IRMultimedia Tools and Applications10.1007/s11042-017-4997-y77:11(13853-13889)Online publication date: 1-Jun-2018
    • (2017)Learning Social Circles in Ego-Networks Based on Multi-View Network StructureIEEE Transactions on Knowledge and Data Engineering10.1109/TKDE.2017.268538529:8(1681-1694)Online publication date: 1-Aug-2017
    • (2017)ACSIR: ANOVA Cosine Similarity Image Recommendation in vertical searchInternational Journal of Multimedia Information Retrieval10.1007/s13735-017-0124-06:2(143-154)Online publication date: 31-Mar-2017
    • (2017)Fusion Strategies for Large-Scale Multi-modal Image RetrievalTransactions on Large-Scale Data- and Knowledge-Centered Systems XXXIII10.1007/978-3-662-55696-2_5(146-184)Online publication date: 8-Aug-2017
    • (2016)Image Recommendation Based on ANOVA Cosine SimilarityProcedia Computer Science10.1016/j.procs.2016.06.09189(562-567)Online publication date: 2016
    • (2013)When Amazon Meets GoogleACM Transactions on Internet Technology10.1145/249269012:4(1-17)Online publication date: 1-Jul-2013
    • (2013)Personal Clothing Retrieval on Photo Collections by Color and AttributesIEEE Transactions on Multimedia10.1109/TMM.2013.227965815:8(2035-2045)Online publication date: Dec-2013
    • (2013)iLikeIEEE Transactions on Knowledge and Data Engineering10.1109/TKDE.2012.19225:10(2257-2270)Online publication date: 1-Oct-2013
    • (2013)Multimodal re-ranking of product image search resultsProceedings of the 35th European conference on Advances in Information Retrieval10.1007/978-3-642-36973-5_6(62-73)Online publication date: 24-Mar-2013
    • (2013)Clothing Extraction by Coarse Region Localization and Fine Foreground/Background EstimationAdvances in Multimedia Modeling10.1007/978-3-642-35728-2_30(316-326)Online publication date: 2013
    • Show More Cited By

    View Options

    Login options

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Figures

    Tables

    Media

    Share

    Share

    Share this Publication link

    Share on social media