Abstract
In this chapter, we present an approach to handle multi–modality in image retrieval using a Vector Space Model (VSM), which is extensively used in text retrieval. We simply extended the model with visual terms aiming to close the semantic gap by helping to map low–level features into high level textual semantic concepts. Moreover, this combination of textual and visual modality into one space also helps to query a textual database with visual content, or a visual database with textual content. Alongside this, in order to improve the performance of text retrieval we propose a novel expansion and re–ranking method, applied both to the documents and the query. When textual annotations of images are acquired automatically, they may contain too much information, and document expansion adds more noise to retrieval results. We propose a re–ranking phase to discard such noisy terms. The approaches introduced in this chapter were evaluated in two sub–tasks of ImageCLEF2009. First, we tested the multi–modality part in ImageCLEFmed and obtained the best rank in mixed retrieval, which includes textual and visual modalities. Secondly, we tested expansion and re–ranking methods in ImageCLEFWiki and the results were superior to others and obtained the best four positions in text–only retrieval. The results showed that the handling of multi–modality in text retrieval using a VSM is promising, and document expansion and re–ranking plays an important role in text–based image retrieval.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Allan J, Leuski A, Swan R, Byrd D (2001) Evaluating combinations of ranked lists and visualizations of inter–document similarity. Information Processing & Management 37(3):435–458
Amati G, Van Rijsbergen CJ (2002) Probabilistic models of information retrieval based on measuring the divergence from randomness. ACM Transactions on Information Systems 20(4):357–389
Balinski J, Danilowicz C (2005) Re–ranking method based on inter–document distances. Information Processing & Management 41(4):759–775
Beaulieu MM, Gatford M, Xiangji H, Robertson SE, Walker S, Williams P (1997) Okapi at TREC–5. In: Proceedings of the Fifth Text REtrieval Conference (TREC–5), National Institute of Standards and Technology, 500238, pp 143–165
Billerbeck B, Zobel J (2005) Document expansion versus query expansion for ad–hoc retrieval. In: Proceedings of the Tenth Australasian Document Computing Symposium, pp 34–41
Buckley C (1993) The importance of proper weighting methods. In: Proceedings of the workshop on Human Language Technology, Association for Computational Linguistics, pp 349–352
Callan J, Croft WB, Harding SM (1992) The INQUERY retrieval system. In: In Proceedings of the Third International Conference on Database and Expert Systems Applications, Springer, pp 78–83
Can F, Ozkarahan EA (1990) Concepts and effectiveness of the cover–coefficient–based clustering methodology for text databases. ACM Transactions of Database Systems 15(4):483–517
Carbonell J, Goldstein J (1998) The use of mmr, diversity–based reranking for reordering documents and producing summaries. In: Proceedings of the 21st annual international ACM SIGIR conference on research and development in information retrieval, ACM press, pp 335–336
Chang N, Fu K (1980) Query–by–pictorial–example. IEEE Transactions on Software Engineering 6:519–524
Chang SK, Kunil TL (1981) Pictorial data–base systems. Computer 14(11):13–21
Chisholm E, Kolda TG (1999) New term weighting formulas for the vector space method in information retrieval. Tech. rep., Oak Ridge National Laboratory
El-Kwae EA, Kabuka MR (2000) Efficient content–based indexing of large image databases. ACM Transactions on Information Systems 18(2):171–210
Flickner M, Sawhney H, Niblack W, Ashley J, Huang Q, Dom B, Gorkani M, Hafner J, Lee D, Petkovic D, Steele D, Yanker P (1995) Query by image and video content: The QBIC system. Computer 28(9):23–32
Frankel C, Swain MJ, Athitsos V (1996) Webseer: An image search engine for the world wide web. Tech. rep., University of Chicago, Chicago, IL, USA
Jain R, Lew MS, Lempinen K, Huijsmans N (1997) Webcrawling using sketches
Lee KS, Park YC, Choi KS (2001) Re–ranking model based on document clusters. Information Processing & Management 37(1):1–14
Lesk M (1986) Automatic sense disambiguation using machine readable dictionaries: how to tell a pine cone from an ice cream cone. In: Proceedings of the 5th annual international conference on Systems documentation, ACM press, pp 24–26
Li Y, Bandar ZA, McLean D (2003) An approach for measuring semantic similarity between words using multiple information sources. IEEE Transactions On Knowledge and Data Engineering
Lingpeng Y, Donghong J, Guodong Z, Yu N (2005) Improving retrieval effectiveness by using key terms in top retrieved documents. Advances in Information Retrieval 169–184
Manning CD, Raghavan P, Schtze H (2008) Introduction to information retrieval. Cambridge University Press, New York, NY, USA
Miller GA, Beckwith R, Fellbaum C, Gross D, Miller KJ (1990) Introduction to wordnet: An on-line lexical database. International Journal of Lexicography 3(4):235–244
Müller H, Kalpathy-Cramer J, Eggel I, Bedrick S, Radhouani S, Bakke B, Jr. C, Hersh W (2009) Overview of the ImageCLEF 2009 medical image retrieval track. In: CLEF working notes 2009
Ogle VE, Stonebraker M (1995) Chabot: Retrieval from a relational database of images. Computer 28:40–48
Resnik P (1999) Semantic similarity in a taxonomy: An information–based measure and its application to problems of ambiguity in natural language. Journal of Artificial Intelligence Research 11:95–130
Richardson R, Smeaton AF, Murphy J (1994) Using wordnet as a knowledge base for measuring semantic similarity between words. In: In Proceedings of Irish Conference on Artificial Intelligence and Cognitive Science
Salton G, Wong A, Yang C (1975) A vector space model for information retrieval. Journal of the American Society for Information Science 18(11):613–620
Santini S, Jain R (2000) Integrated browsing and querying for image databases. IEEE MultiMedia 7:26–39
Singhal A, Pereira F (1999) Document expansion for speech retrieval. In: Proceedings of the 22nd annual international ACM SIGIR conference on research and development in information retrieval, ACM press, pp 34–41
Singhal A, Buckley C, Mitra M (1996) Pivoted document length normalization. In: Proceedings of the 19th annual international ACM SIGIR conference on research and development in information retrieval, ACM press
Tsikrika T, Kludas J (2009) Overview of the wikipediamm task at ImageCLEF 2009. In: Working notes CLEF 2009, Corfu, Greece
Tversky A (1977) Features of similarity. Psychological Review 84(4):327–352
Wong SKM, Yao YY (1995) On modeling information retrieval with probabilistic inference. ACM Transactions on Information Systems 13(1):38–68
Wu JK (1997) Content–based indexing of multimedia databases. IEEE Transactions on Knowledge and Data Engineering 9:978–989
Wu Q, Iyengar SS, Zhu M (2001) Web image retrieval using self–organizing feature map. Journal of the American Society for Information Science and Technology 52(10):868–875
Wu Z, Palmer M (1994) Verbs semantics and lexical selection. In: Proceedings of the 32nd annual meeting on Association for Computational Linguistics, Association for Computational Linguistics, pp 133–138
Yang L, Ji D, Zhou G, Nie Y, Xiao G (2006) Document re–ranking using cluster validation and label propagation. In: Proceedings of the 15th ACM international conference on Information and knowledge management, ACM press, pp 690–697
Zhang R, Chang Y, Zheng Z, Metzler D, Nie Jy (2009) Search result re–ranking by feedback control adjustment for time-sensitive query. In: Proceedings of Human Language Technologies: The 2009 Annual Conference of the North American Chapter of the Association for Computational Linguistics, Companion Volume: Short Papers, Association for Computational Linguistics, pp 165–168
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2010 Springer-Verlag Berlin Heidelberg
About this chapter
Cite this chapter
Alpkocak, A., Kilinc, D., Berber, T. (2010). Expansion and Re–ranking Approaches for Multimodal Image Retrieval using Text–based Methods. In: Müller, H., Clough, P., Deselaers, T., Caputo, B. (eds) ImageCLEF. The Information Retrieval Series, vol 32. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-15181-1_14
Download citation
DOI: https://doi.org/10.1007/978-3-642-15181-1_14
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-15180-4
Online ISBN: 978-3-642-15181-1
eBook Packages: Computer ScienceComputer Science (R0)