Abstract
Nowadays, more and more people prefer online shopping to physical store shopping for its convenience, cheapness and timesaving. Customers visit some commercial shopping websites, and select their favorite commodities by accessing links or retrieving by search box. However, in our real life, most online shopping websites provide a simple and single text retrieval method only, to some extent it’s difficult for customers to submit query and retrieve satisfactory results. In this paper, a multi-modal search approach combining text-based and image-based search techniques is presented. Besides text search, a two-stage image search approach is proposed, which utilizes basic features consisting of color and textural features to filter mismatching images in first stage, and further uses SIFT features for accurate search in second stage. Moreover, a prototype system has been developed for multi-modal search on online shopping websites. By submitting some words, phrases, images or their combination, customers can search out what they want. The experiments compared with traditional algorithms based on single visual feature validate that our approach and multi-modal search prototype system are effective, and the retrieval results can satisfy customers’ requirements well for online shopping.
Project supported by the State Key Development Program for Basic Research of China (Grant No. 2011CB302200-G), National Natural Science Foundation of China (Grant No. 60973019, 61100026), and the Fundamental Research Funds for the Central Universities(N100704001).
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
TNS Interactive. Global e-Commerce Report (2002), http://www.tnsofres.com/ger2002/ (accessed July 5, 2004)
Michael, R., Michelle, M.: Consumer acquisition of product information and subsequent purchase channel decisions. Volume Advances in Applied Microeconomics Issue, 231–255 (2002)
Couclelis, H.: Pizza over the Internet: E-commerce, the fragmentation of activity and the tyranny of the region. Entrepreneurship & Regional Development 16, 41–54 (2004)
Davis, Z., Hu, M., et al.: A Personal Handheld Multi-Modal Shopping Assistant. Networking and Service, 117–125 (2006)
Anil, K., Jain, A.: Shape-Based Retrieval: A Case Study With Trademark Image Databases. Pattern Recognition (PR) 31(9), 1369–1390 (1998)
Liu, Y., Zhang, D., Lu, G., Ma, W.: A survey of content-based image retrieval with high-level semantics. Pattern Recognition (PR) 40(1), 262–282 (2007)
Flickner, M., Sawhney, H., et al.: Query by Image and Video Content: The QBIC System. IEEE Computer 28(9), 23–32 (1995)
Bach, J., Fuller, C., et al.: Virage Image Search Engine: An Open Framework for Image Management. In: Storage and Retrieval for Image and Video Databases (SPIE), pp. 76–87 (1996)
Huang, T., Mehrotra, S., Ramchandran, K.: Multimedia Analysis and Retrieval System (MARS) Project. Data Processing Clinic (1996)
Smith, J.: Integrated spatial and feature image systems: Retrieval, compression and analysis, Ph.D. dissertation, Columbia Univ., NewYork (1997)
Song, K., Kittler, J., Petrou, M.: Defect detection in random color textures. Image and Vision Computing, 667–684 (1996)
Baraldi, A., Parmiggiani, F.: An investigation of the textural characteristics associated with gray level cooccurrence matrix statistical parameters. Geoscience and Remote Sensing Society 33(2), 193–304 (2002)
Malik, J., Belongie, S., Leung, T., Shi, J.: Contour and Texture Analysis for Image Segmentation. International Journal of Computer Vision (IJCV) 43(1), 7–27 (2001)
Lowe, D.: Distinctive Image Features from Scale-Invariant Keypoints. International Journal of Computer Vision (IJCV) 60(2), 91–110 (2004)
Indyk, P., Motwani, R.: Approximate Nearest Neighbors: Towards Removing the Curse of Dimensionality. In: STOC, pp. 604–613 (1998)
Gionis, A., Indyk, P., Motwani, R.: Similarity Search in High Dimensions via Hashing. In: VLDB, pp. 518–529 (1999)
Datar, M., Immorlica, N., Indyk, P., Mirrokni, V.: Locality-sensitive hashing scheme based on p-stable distributions. In: SoCG, pp. 253–262 (2004)
Skrypnyk, L.D.: Scene Modelling, Recognition and Tracking with Invariant Image Features. In: ISMAR, pp. 110–119 (2004)
Doug Cutting. Lucene, http://lucene.apache.org/
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2012 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Li, R., Wang, D., Zhang, Y., Feng, S., Yu, G. (2012). An Approach of Text-Based and Image-Based Multi-modal Search for Online Shopping. In: Gao, H., Lim, L., Wang, W., Li, C., Chen, L. (eds) Web-Age Information Management. WAIM 2012. Lecture Notes in Computer Science, vol 7418. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-32281-5_17
Download citation
DOI: https://doi.org/10.1007/978-3-642-32281-5_17
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-32280-8
Online ISBN: 978-3-642-32281-5
eBook Packages: Computer ScienceComputer Science (R0)