Abstract
This paper presents a novel approach for searching images online using textual queries and presenting the resulting images based on both conceptual and visual similarities. Given a user-specified query, the algorithm first finds the related concepts through conceptual query expansion. Each concept, together with the original query, is then used to search for images using existing image search engines. All the images found under different concepts are presented on a 2D virtual canvas using a self-organizing map. Both conceptual and visual similarities among the images are used to determine the image locations so that images from the same or related concepts are grouped together and visually similar images are placed close to each other. When the user browses the search results, a subset of representative images is selected to compose an image collage. Once having identified images of interest within the collage, the user can find more images that are conceptually or visually similar through pan and zoom operations. Experiments on different image query examples demonstrate the effectiveness of the presented approach.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Algorithmics Group: MDSJ: Java Library for Multidimensional Scaling, Version 0.2 (2009), http://www.inf.uni-konstanz.de/algo/software/mdsj/
André, P., Cutrell, E., Tan, D.S., Smith, G.: Designing novel image search interfaces by understanding unique characteristics and usage. In: Proc. IFIP Conference on Human-Computer Interaction, pp. 340–353 (2009)
Borg, I., Groenen, P.: Modern Multidimensional Scaling: Theory and Applications, 2nd edn. Springer, Heidelberg (2005)
Datta, R., Joshi, D., Li, J., Wang, J.Z.: Image Retrieval: Ideas, Influences, and Trends of the New Age. ACM Computing Surveys 40(2), 1–60 (2008)
Fonseca, B.M., Golgher, P., Pôssas, B., Ribeiro-Neto, B., Ziviani, N.: Concept-based interactive query expansion. In: Proc. ACM International Conference on Information and Knowledge Management, pp. 696–703 (2005)
Gabrilovich, E., Markovitch, S.: Computing semantic relatedness using wikipedia-based explicit semantic analysis. In: Proc. International Joint Conference on Artificial Intelligence, pp. 1606–1611 (2007)
Google: Google Image Swirl (2009), http://image-swirl.googlelabs.com/
Heesch, D.: A survey of browsing models for content based image retrieval. Multimedia Tools and Applications 40(2), 261–284 (2008)
Heesch, D., Rüger, S.: Image Browsing: A semantic analysis of NNk networks. In: Proc. International Conference Image and Video Retrieval, pp. 609–618 (2005)
Jansen, B.J., Spink, A., Pedersen, J.: An analysis of multimedia searching on AltaVista. In: Proc. ACM SIGMM International Workshop on Multimedia Information Retrieval, pp. 186–192 (2003)
Joshi, D., Datta, R., Zhuang, Z., Weiss, W.P., Friedenberg, M., Li, J., Wang, J.Z.: PARAgrab: A comprehensive architecture for web image management and multimodal querying. In: Proc. International Conference on Very Large Databases, pp. 1163–1166 (2006)
Kherfi, M.L., Ziou, D., Bernardi, A.: Image Retrieval from the World Wide Web: Issues, Techniques, and Systems. ACM Computer Survey 36(1), 35–67 (2004)
Milne, D., Witten, I.H.: An effective, low-cost measure of semantic relatedness obtained from wikipedia links. In: Proc. AAAI Workshop on Wikipedia and Artificial Intelligence, pp. 25–30 (2008)
Myoupo, D., Popescu, A., Borgne, H.L., Moëllic, P.A.: Multimodal image retrieval over a large database. In: Peters, C., Caputo, B., Gonzalo, J., Jones, G.J.F., Kalpathy-Cramer, J., Müller, H., Tsikrika, T. (eds.) CLEF 2009 Workshop, Part II. LNCS, vol. 6242, pp. 1–8. Springer, Heidelberg (2010)
Nguyen, G.P., Worring, M.: Interactive access to large image collections using similarity-based visualization. J. Vis. Lang. Comput. 19(2), 203–224 (2008)
Pečenović, Z., Do, M., Vetterli, M., Pu, P.: Integrated Browsing and Searching of Large Image Collections. In: Laurini, R. (ed.) VISUAL 2000. LNCS, vol. 1929, pp. 279–289. Springer, Heidelberg (2000)
Smeulders, A.W.M., Worring, M., Santini, S., Gupta, A., Jain, R.: Content-based image retrieval at the end of the early years. IEEE Transactions on Pattern Analysis and Machine Intelligence 22(12), 1349–1380 (2000)
Strong, G., Gong, M.: Browsing a large collection of community photos based on similarity on GPU. In: Bebis, G., Boyle, R., Parvin, B., Koracin, D., Remagnino, P., Porikli, F., Peters, J., Klosowski, J., Arns, L., Chun, Y.K., Rhyne, T.-M., Monroe, L. (eds.) ISVC 2008, Part II. LNCS, vol. 5359, pp. 390–399. Springer, Heidelberg (2008)
Strong, G., Gong, M.: Organizing and Browsing Photos Using Different Feature Vectors and Their Evaluations. In: Proc. International Conference on Image and Video Retrieval, pp. 1–8 (2009)
Strong, G., Hoeber, O., Gong, M.: Visual image browsing and exploration (vibe): user evaluations of image search tasks. In: Proc. International Conference on Active Media Technology, pp. 424–435 (2010)
Strube, M., Ponzetto, S.P.: WikiRelate! computing semantic relatedness using wikipedia. In: Proc. AAAI Conference on Artificial Intelligence, pp. 1419–1424 (2006)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2010 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Strong, G., Hoque, E., Gong, M., Hoeber, O. (2010). Organizing and Browsing Image Search Results Based on Conceptual and Visual Similarities. In: Bebis, G., et al. Advances in Visual Computing. ISVC 2010. Lecture Notes in Computer Science, vol 6454. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-17274-8_47
Download citation
DOI: https://doi.org/10.1007/978-3-642-17274-8_47
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-17273-1
Online ISBN: 978-3-642-17274-8
eBook Packages: Computer ScienceComputer Science (R0)