Abstract.
Content-based image retrieval (CBIR) has become one of the most active research areas in the past few years. Many visual feature representations have been explored and many systems built. In this paper, we focus on an important component of these systems - relevance feedback - and how we incorporated it into the MARS retrieval system. Relevance feedback techniques are based on an interactive retrieval approach to effectively take into account user preferences to provide an improved search experience. We present a series of coherent strategies, from single-point to multipoint and multifeature approaches that we have seamlessly integrated into our system and present experimental results to show their retrieval performance characteristics.
Keywords: Image retrieval - Query refinement - Relevance feedback
Similar content being viewed by others
References
Baeza-Yates R, Ribiero-Neto B (1999) Modern information retrieval. ACM Press Series/Addison-Wesley, New York
Bartolini I, Ciaccia P, Waas F (/2001) Feedback bypass: a new approach to interactive similarity query processing. In: Proceedings of the 27th conference on very large databases (VLDB), Rome, September 2001, pp 201-210
Carmel D, Farchi E, Petruschka Y, Soffer A (2002) Automatic query refinement using lexical affinities with maximal information gain. In: ACM SIGIR, Tampere, Finland, 11-15 August 2002, pp 283-290
Chang E, Li B (2002) MEGA - the maximizing expected generalization algorithm for learning complex query concepts. ACM Trans Inf Sys 21(4)
Chang T, Jay Kuo C-C (1993) Texture analysis and classification with tree-structured wavelet transform. IEEE Trans Image Proc 2(4):429-441
Chen Y, Zhou X, Huang TS (1999) One-class SVM for learning in image retrieval. In: Proceedings of the IEEE international conference on image processing, Kobe, Japan, February 1999, pp 440-447
Cheng B (1996) Approaches to image retrieval based on compressed data for multimedia database systems. PhD Thesis, State University of New York at Buffalo
Equitz W, Niblack W (1994) Retrieving images from a database using texture - algorithms from the QBIC system. Technical Report RJ 9805, Computer Science, IBM Research Report, May 1994
Flickner M, Sawhney H, Niblack W, Ashley J, Huang Q, Dom B, Gorkani M, Hafine J, Lee D, Petkovic D, Steele D, Yanker P (1995) Query by image and video content: the QBIC system. IEEE Comput 28(9):23-32
Gross MH, Koch R, Lippert L, Dreger A (1994) Multiscale image texture analysis in wavelet spaces. In: Proceedings of the IEEE international conference on image processing, Austin, TX, November, 1994, pp 412-416
Heisterkamp D, Peng J, Dai HK (2001) Adaptive quasiconformal kernel metric for image retrieval. In: Proceedings of the IEEE conference on computer vision and pattern recognition, Kauai, HI, December 2001, pp 236-243
Huang TS, Mehrotra S, Ramchandran K (1996) Multimedia analysis and retrieval system (MARS). In: Proceedings of the 33rd annual clinic on library application of data processing, “Digital Image Access and Retrieval”, Urbana-Champaign, IL, March 1996
Ishikawa Y, Subramanya R, Faloutsos C (1998) Mindreader: querying databases through multiple examples. In: Proceedings of the conference on very large databases, New York, August 1998, pp 218-227
Jansen BJ, Spink A, Saracevic T (2000) Real life, real users, and real needs: a study and analysis of user queries on the web. Inf Process Manage 36(2):207-227
Kohrs A, Merialdo B (1999) Improving collaborative filtering with multimedia indexing techniques to create user-adaptive Web sites. In: Proceedings of the ACM multimedia conference, Orlando, FL, October/November 1999, pp 27-36
Liu F, Picard RW (1996) Periodicity, directionality, and randomness: Wold features for image modeling and retrieval. IEEE Trans Patt Recog Mach Intell 18(7):722-733
Ma WY, Manjunath BS (1995) A comparison of wavelet transform features for texture image annotation. In: Proceedings of the IEEE international conference on image processing, Washington, DC, October 1995, pp 256-259
Ma WY, Manjunath BS (1997) Netra: a toolbox for navigating large image databases. In: Proceedings of the IEEE international conference on image processing, Santa Barbara, CA, October 1997, pp 568-571
Mandal MK, Aboulnasr T, Panchanathan S (1996) Image indexing using moments and wavelets. IEEE Trans Consumer Electron 42(3):557-565
Manjunath BS, Ma WY (1996) Texture features for browsing and retrieval of image data. In: IEEE T-PAMI special issue on digital libraries, November 1996
Mao J, Jain AK (1992) Texture classification and segmentation using multiresolution simultaneous autoregressive models. Patt Recog 25(2):173-188
Mehrotra S, Chakrabarti K, Ortega M, Rui Y, Huang TS (1997) Towards extending information retrieval techniques for multimedia retrieval. In: Proceedings of the 3rd international workshop on multimedia information systems, Como, Italy, September 1997
Minka TP, Picard RW (1996) Interactive learning using a “society of models”. In: Proceedings of IEEE CVPR, San Francisco, June, 1996, pp 447-452
Ortega M, Rui Y, Chakrabarti K, Porkaew K, Mehrotra S, Huang TS (1998) Supporting ranked boolean similarity queries in mars. IEEE Trans Data Eng 10(6):905-925
Ortega-Binderberger M, Chakrabarti K, Mehrotra S (2002) An approach to integrating query refinement in SQL. In: Proceedings of EDBT, Prague, Czech Republic, March 2002, pp 15-33
Pentland A, Picard RW, Sclaroff S (1994) Photobook: Tools for content-based manipulation of image databases. In: Proceedings of the 2nd SPIE conference on storage and retrieval for image and video databases, Bellingham, WA, 2(185):34-47
Porkaew K, Mehrotra S, Ortega M (1999a) Query reformulation for content based multimedia retrieval in mars. In: Proceedings of the IEEE international conference on multimedia computing and systems (ICMCS 99), June 1999
Porkaew K, Mehrotra S, Ortega M, Chakrabarti K (1999b) Similarity search using multiple examples in mars. In: Proceedings of Visual’99, June 1999
Rocchio JJ (1971) Relevance feedback in information retrieval. In: Salton G (ed) The SMART retrieval system. Prentice-Hall, Englewood Cliffs, NJ, pp 313-323
Rui Y, She AC, Huang TS (1996) Automated region segmentation using attraction-based grouping in spatial-color-texture space. In: Proceedings of the international conference on image processing (ICIP’96), Lausanne, Switzerland, September 1996, pp 53-56
Rui Y, Huang TS, Mehrotra S, Ortega M (1997) Automatic matching tool selection via relevance feedback in MARS. In: Proceedings of the 2nd international conference on visual information systems, San Diego, December 1997, pp 109-116
Rui Y, Huang TS, Ortega M, Mehrotra S (1998) Relevance feedback: a power tool for interactive content-based image retrieval. IEEE Trans Circuits Video Technol, 8(5)664-655
Salton G, McGill MJ (1983) Introduction to modern information retrieval. McGraw-Hill, New York
Sclaroff S, Taycher L, La Cascia M (1997) Imagerover: a content-based image browser for the World Wide Web. In: Proceedings of the IEEE workshop on content-based access of image and video libraries, Puerto Rico, June 1997, pp 2-9
Shafer JC, Agrawal R (2000) Continuous querying in database-centric web applications. In: Proceedings of the WWW9 conference, Amsterdam, May 2000
Smith JR, Chang S-F (1994) Transform features for texture classification and discrimination in large image databases. In: Proceedings of the IEEE international conference on image processing, Austin, TX, November 1994, pp 407-411
Spink A, Saracevic T (1998) Human-computer interaction in information retrieval: nature and manifestations of feedback. Interact Comput 10(3):249-267
Tamura H(1978) Texture features corresponding to visual perception. In: IEEE Trans Sys Man Cybern SMC-8(6):460-473
Valiant LG (1984) A theory of the learnable. Commun ACM 27(11):1134-1142
Wu L, Faloutsos C, Sycara K, Payne T (2000) FALCON: feedback adaptive loop for content-based retrieval. In: Proceedings of the conference on very large databases, Cairo, Egypt, September 2000, pp 297-306
Author information
Authors and Affiliations
Corresponding author
Additional information
Michael Ortega-Binderberger: michaelo@us.ibm.comThis work was performed while the author was a Ph.D. student at the University of Illinois at Urbana-Champaign. Correspondence to:
This material is based on work supported in part by the National Science Foundation under Award Numbers CAREER IIS-9734300, 9996140, 0083489, 0331707, and 0331690 and in part by the Army Research Laboratory under Cooperative Agreement No. DAAL01-96-2-0003. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of the National Science Foundation or the Army Research Laboratory. Michael Ortega-Binderberger was supported in part by CONACYT award # 89061.
Rights and permissions
About this article
Cite this article
Ortega-Binderberger, M., Mehrotra, S. Relevance feedback techniques in the MARS image retrieval system. Multimedia Systems 9, 535–547 (2004). https://doi.org/10.1007/s00530-003-0126-z
Issue Date:
DOI: https://doi.org/10.1007/s00530-003-0126-z