Abstract
In this paper, we introduce a novel approach to image-based information retrieval by combining image analysis with linguistic analysis of associated annotation information. While numerous Content Based Image Retrieval (CBIR) systems exist, most of them are constrained to use images as the only source of information. In contrast, recent research, especially in the area of web-search has also used techniques that rely purely on textual information associated with an image. The proposed research adopts a conceptually different philosophy. It utilizes the information at both the image and annotation level, if it detects a strong semantic coherence between them. Otherwise, depending on the quality of information available, either of the media is selected to execute the search. Semantic similarity is defined through the use of linguistic relationships in WordNet as well as through shape, texture, and color. Our investigations lead to results that are of significance in designing multimedia information retrieval systems. These include technical details on designing cross-media retrieval strategies as well as the conclusion that combining information modalities during retrieval not only leads to more semantically relevant performance but can also help capture highly complex issues such as the emergent semantics associated with images.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Aslandogan, Y., Their, C., Yu, C., Zou, J., Rishe, N.: Using Semantic Contents and WordNet in Image Retrieval. In: Proceedings of ACM SIGIR Conference, Philadelphia (July 1997)
Barnard, K., Forsyth, D.: Learning the Semantics of Words and Pictures. In: International Conference on Computer Vision, vol. 2, pp. 408–415 (2001)
Carson, C., Belonge, S., Greenspan, H., Malik, J.: Blobworld: Image segmentation using Expectation-Maximization and its application to image querying. IEEE Transactions on Pattern Analysis and Machine Intelligence, SUB
Chen, F., Gargi, U., Niles, L., Schütze, H.: Multi-modal browsing of images in web documents. In: Proc. SPIE Document Recognition and Retrieval (1999)
La Cascia, M., Sethi, S., Sclaroff, S.: Combining Textual and Visual Cues for Content-based Image Retrieval on the World Wide Web. In: IEEE Workshop on Content-based Access of Image and Video Libraries
Deng, C., He, X., Li, Z., Ma, W., Wen, J.: Hierarchical Clustering of WWW Image Search Results Using Visual, Textual and Link Information. In: Proceedings of the 12th annual ACM international conference on Multimedia, pp. 952–959 (2004)
Deng, Y., Manjunath, B., Kenney, C., Moore, M., Shin, H.: An Efficient Color Representation for Image Retrieval. IEEE Transactions on Image Processing 10(1), 140–147 (2001)
Flickr, http://www.flickr.com/
Google search engine, http://www.google.com/
Jacobs, C., Finkelstein, A., Salesin, D.: Fast Multiresolution Image Querying. In: Proceedings of Computer Graphics, Annual Conference Series, pp. 277–286 (1995)
Miller, G., Beckwith, R., Fellbaum, C., Gross, D., Miller, K.: Introduction to WordNet: An on-line lexical database. International Journal of Lexicography 3(4), 235–312 (1990)
Paek, S., Sable, C.L., Hatzivassiloglou, V., Jaimes, A., Schiffman, B.H., Chang, S.-F., McKeown, K.R.: Integration of Visual and Text based Approaches for the Content Labeling and 21 Classification of Photographs. In: ACM SIGIR 1999 Workshop on Multimedia Indexing and Retrieval (1999)
Rodden, K., Basalaj, W., Sinclair, D., Wood, K.R.: Does organisation by similarity assist image browsing? In: Proceedings of Human Factors in Computing Systems (2001)
Sable, C., Hatzivassiloglou, V.: Text-based approaches for the categorization of images. In: Abiteboul, S., Vercoustre, A.-M. (eds.) ECDL 1999. LNCS, vol. 1696, pp. 19–38. Springer, Heidelberg (1999)
Santini, S., Gupta, A., Jain, R.: Emergent Semantics Through Interaction in Image Databases. Knowledge and Data Engineering 13(3), 337–351 (2001)
Sclaroff, S., Taycher, L., La Cascia, M.: ImageRover: A Content-Based Image Browser for the World Wide Web. In: IEEE Workshop on Content-based Access of Image and Video Libraries, TR97-005 06/97
Wang, J., Li, J., Wiederhold, G.: Semantics-Sensitive Integrated Matching for Picture Libraries. IEEE Transactions on Pattern Analysis and Machine Intelligence 23(9), 947–963 (2001)
Wang, J., Wiederhold, G., Firschein, O., Wei, S.: Content-based image indexing and searching using daubechies’ wavelets. International Journal of Digital Libraries 1(4), 311–328 (1998)
Yee, K., Swearingen, K., Li, K., Heart, M.: Faceted Metadata for Image Search and Browsing. In: Proceedings of the Conference on Human Factors in Computing Systems, pp. 401–408 (2003)
Zambrano, B., Singh, R., Bhattarai, B.: Using Linguistic Models for Image Retrieval. In: Bebis, G., Boyle, R., Koracin, D., Parvin, B. (eds.) ISVC 2005. LNCS, vol. 3804, pp. 494–501. Springer, Heidelberg (2005)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2006 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Lam, T., Singh, R. (2006). Semantically Relevant Image Retrieval by Combining Image and Linguistic Analysis. In: Bebis, G., et al. Advances in Visual Computing. ISVC 2006. Lecture Notes in Computer Science, vol 4292. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11919629_77
Download citation
DOI: https://doi.org/10.1007/11919629_77
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-48626-8
Online ISBN: 978-3-540-48627-5
eBook Packages: Computer ScienceComputer Science (R0)