Abstract
In content-based image retrieval, relevance feedback is an interactive process, which builds a bridge to connect users with a search engine. It leads to much improved retrieval performance by updating a query and similarity measures according to a user’s preference; and recently techniques have matured to some extent. Most previous relevance feedback approaches exploit short-term learning (intraquery learning) that deals with the current feedback session but ignoring historical data from other users, which potentially results in a great loss of useful information. In the last few years, long-term learning (inter-query learning), by recording and collecting feedback knowledge from different users over a variety of query sessions has played an increasingly important role in multimedia information searching. It can further improve the retrieval performance in terms of effectiveness and efficiency. In the published literature, no comprehensive survey of both short-term learning and long-term learning RF techniques has been conducted. To this end, the goal of this chapter is to address this omission and offer suggestions for future work.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Anick, P.: Using Terminological Feedback for Web Search Refinement - A Log-based Study. In: Proc. Int’l ACM SIGIR Conf. Research and Development in Information Retrieval, pp. 88–95 (2003)
Bian, W., Tao, D.: Biased Discriminant Euclidean Embedding for Content-Based Image Retrieval. IEEE Transactions on Image Processing 19(2), 545–554 (2010)
Blum, A., Mitchell, T.: Combining Labeled and Unlabeled Data with Cotraining. In: COLT: Proc. Workshop on Computational Learning Theory (1998)
Breiman, L.: Bagging Predictors. Machine Learning 24(2), 123–140 (1995)
Burges, J.C.: A Tutorial on Support Vector Machines for Pattern Recognition. Data Mining and Knowledge Discovery 2(2), 121–167 (1998)
Carson, C., Belongie, S., Greenspan, H., Malik, J.: Blobworld: Image Segmentation Using Expectation-Maximization and Its Application to Image Querying. IEEE Trans. Pattern Analysis and Machine Intelligence 24(8), 1026–1038 (2002)
Chang, T., Kuo, C.: Texture Analysis and Classification with Tree-Structured Wavelet Transform. IEEE Trans. Image Processing 2(4), 429–441 (1993)
Chang, H., Yeung, D.Y.: Stepwise Metric Adaptation Based on Semi-Supervised Learning for Boosting Image Retrieval Performance. In: British Machine Vision Conference (2005)
Chen, X., Zhang, C., Chen, S.C., Chen, M.: A Latent Semantic Indexing Based Method for Solving Multiple Instance Learning Problem in Region-Based Image Retrieval. In: Proc. IEEE Int’l Symposium Multimedia, pp. 37–45 (2005)
Chen, Y., Zhou, X., Huang, T.S.: One-class SVM for Learning in Image Retrieval. In: Proc. IEEE Int’l Conf. Image Processing, vol. 1, pp. 34–37 (2001)
Cord, M., Gosselin, P.H.: Image Retrieval using Long-Term Semantic Learning. In: IEEE Int’l Conf. Image Processing, pp. 2,909–2,912 (2006)
Cox, I.J., Miller, M.L., Minka, T.P., Papathomas, T.V., Yianilos, P.N.: The Bayesian Image Retrieval System, PicHunter: Theory, Implementation and Psychophysical Experiments. IEEE Trans. Image Processing 9(1), 20–37 (2000)
Cui, H., Wen, J., Nie, J., Ma, W.: Query Expansion by Mining User Logs. IEEE Trans. Knowledge and Data Engineering 15(4), 829–939 (2003)
Daubechies, I.: The Wavelet Transform, Time-frequency Localization and Signal Analysis. IEEE Trans. Information Theory 36(5), 961–1005 (1990)
Daugman, J.G.: Two-Dimensional Spectral Analysis of Cortical Receptive Field Profile. Vision Research 20, 847–856 (1980)
Daugman, J.G.: Uncertainty Relation for Resolution in Space, Spatial Frequency, and Orientation Optimized by Two–Dimensional Visual Cortical Filters. J. Optical Society of America 2(7), 1,160–1,169 (1985)
Deerwester, S., Dumais, S.T., Furnas, G.W., Landauer, T.K., Harshman, R.: Indexing by latent semantic analysis. J. American Society for Information Science 41, 391–407 (1990)
Deng, J., Dong, W., Socher, R., Li, L.J., Li, K., Li, F.F.: ImageNet: A Large-scale Hierarchical Image Database. In: IEEE Conf. Computer Vision and Pattern Recognition (2009)
Duda, R.O., Hart, P.E., Stork, D.G.: Pattern Classification. Wiley, John & Sons, Incorporated (1999)
Fahlman, S.E., Lebiere, C.: The Cascade-Correlation Learning Architecture. In: Advances in Neural Information Processing Systems 2, pp. 524–532 (1990)
Freund, Y., Schapire, R.E.: Experiments with a New Boosting Algorithm. In: Proc. Int’l Conf. Machine Learning, pp. 148–156 (1996)
Fournier, J., Cord, M.: Long-term Similarity Learning in Content-based Image Retrieval. In: Proc. IEEE Int’l Conf. Image Processing, vol. 1, pp. 1,441–1,444 (2002)
Gevers, T., Smeulders, A.: Pictoseek: Combining color and shape invariant features for image retrieval. IEEE Trans. Image Processing 9(1), 102–119 (2000)
Giacinto, G., Roli, F.: Instance-Based Relevance Feedback for Image Retrieval. In: Advances Neural Information Processing Systems, pp. 489–496 (2004)
Gosselin, P.H., Cord, M.: Semantic Kernel Learning for Interactive Image Retrieval. In: IEEE Int’l Conf. Image Processing, vol. 1, 1,177–1,180 (2005)
Guo, G., Jain, A.K., Ma, W., Zhang, H.: Learning Similarity Measure for Natural Image Retrieval with Relevance Feedback. IEEE Trans. Neural Networks 12(4), 811–820 (2002)
Han, J., Ngan, K.N., Li, M., Zhang, H.J.: A Memory Learning Framework for Effective Image Retrieval. IEEE Trans. Image Processing 14(4), 511–524 (2005)
He, X., King, O., Ma, W., Li, M., Zhang, H.: Learning a Semantic Space from User’s Relevance Feedback for Image Retrieval. IEEE Trans. Circuits and Systems for Video Technology 13(1), 39–48 (2003)
He, X., Ma, W.Y., Zhang, H.J.: Learning an Image Manifold for Retrieval. In: Proc. ACM Int’l Conf. Multimedia, pp. 17–23 (2004)
Heisterkamp, D.R.: Building a Latent Semantic Index of an Image Database from Patterns of Relevance Feedback. In: Proc. Int’l Conf. Pattern Recognition, vol. 4, pp. 134–137 (2002)
Ho, T.K.: The Random Subspace Method for Constructing Decision Forests. IEEE Trans. Pattern Analysis and Machine Intelligence 20(8), 832–844 (1998)
Hoi, C.H., Chan, C.H., Huang, K., Lyu, M.R., King, I.: Biased Support Vector Machine for Relevance Feedback in Image Retrieval. In: Proc. Int’l Joint Conf. Neural Networks, pp. 3189–3194 (2004)
Hoi, C.H., Lyu, M.R.: Group-based Relevance Feedback with Support Vector Machine Ensembles. In: Int’l Conf. Pattern Recognition, vol. 3, pp. 874–877 (2004)
Hoi, S.C.H., Lyu, M.R., Jin, R.: A Unified Log-based Relevance Feedback Scheme for Image Retrieval. IEEE Trans. Knowledge and Data Engineering 18(4), 509–524 (2006)
Hsu, C.T., Li, C.Y.: Relevance Feedback Using Generalized Bayesian Framework With Region-Based Optimization Learning. IEEE Trans. Image Processing 14(10), 1,617–1,631 (2005)
Jain, A., Vailaya, A.: Image Retrieval Using Color and Shape. Pattern Recognition 29(8), 1233–1244 (1996)
Jain, A.K.: Fundamentals of Digital Image Processing. Prentice Hall Inc., New Jersey (1989)
Jain, A.K., Lee, J.E., Jin, R., Gregg, N.: Content-based Image Retrieval: An Application to Tattoo Images. In: IEEE Int’l Conf. Image Processing, vol. 2, pp. 745–742 (2009)
Jiang, W., Er, G., Dai, Q., Gu, J.: Hidden Annotation for Image Retrieval with Long-Term Relevance Feedback Learning. Pattern Recognition 38(11), 2,007–2,021 (2005)
Jing, F., Li, M., Zhang, H., Zhang, B.: Relevance Feedback in Region-Based Image Retrieval. IEEE Trans. Circuits and Systems for Video Technology 14(5), 672–681 (2004)
Koskela, M., Laaksonen, J.: Using Long-Term Learning to Improve Efficiency of Content-Based Image Retrieval. In: Proc. Int’l Workshop on Pattern Recognition in Information Systems, pp. 72–79 (2003)
Kushki, A., Androutsos, P., Plataniotis, K.N., Venetsanopoulos, A.N.: Query Feedback for Interactive Image Retrieval. IEEE Trans. Circuits and Systems for Video Technology 14, 644–655 (2004)
Laaksonen, J., Koskela, M., Laakso, S., Oja, E.: PicSOM—Content-based Image Retrieval with Self-Organizing Maps. Pattern Recognition Letters 21(13-14), 1,199–1,207 (2000)
LeCun, Y., Boser, B., Denker, J.S., Henderson, D., Howard, R.E., Hubbard, W., Jackel, L.D.: Backpropagation Applied to Handwritten Zip Code Recognition. Neural Computation 1(4), 541–551 (1989)
Leung, A.P., Auer, P.: An Efficient Search Algorithm for Content-Based Image Retrieval with User Feedback. In: IEEE Int’l Conf. Data Mining Workshops, pp. 884–890 (2008)
Li, J., Allinson, N.M.: A Comprehensive Review of Current Local Features for Computer Vision. Neurocomputing 71(10-12), 1771–1787 (2008)
Li, J., Allinson, N.M.: Long-term Learning in Content-based Image Retrieval. Int’l J. Imaging Systems and Technology 18(2-3), 160–169 (2008)
Li, J., Allinson, N., Tao, D., Li, X.: Multitraining Support Vector Machine for Image Retrieval. IEEE Trans. Image Processing 15(11), 3,597–3,601 (2006)
Long, F., Zhang, H., Feng, D.: Fundamentals of Content-based Image Retrieval. In: Multimedia Information Retrieval and Management, pp. 1–12. Springer (2003)
Lowe, D.G.: Distinctive Image Features from Scale-Invariant Keypoints. Int’l J. Computer Vision 60(2), 91–110 (2004)
Lu, Y., Zhang, H., Liu, W., Hu, C.: Joint Semantics and Feature Based Image Retrieval Using Relevance Feedback. IEEE Trans. Multimedia 5(3), 339–347 (2003)
Manjunath, B., Ma, W.: Texture Features for Browsing and Retrieval of Image Data. IEEE Trans. Pattern Analysis and Machine Intelligence 18(8), 837–842 (1996)
Manjunath, B., Ohm, J., Vasudevan, V., Yamada, A.: Color and Texture Descriptors. IEEE Trans. Circuits and Systems for Video Technology 11(6), 703–715 (2001)
Mikolajczyk, K., Schmid, C.: A Performance Evaluation of Local Descriptors. IEEE Trans. Pattern Analysis and Machine Intelligence 27(10), 1,615–1,630 (2005)
Mikolajczyk, K., Tuytelaars, T., Schmid, C., Zisserman, A., Matas, J., Schaffalitzky, F., Kadir, T., Van Gool, L.: A Comparison of Affine Region Detectors. Int’l J. Computer Vision 65(1/2), 43–72 (2005)
Minka, T.P., Picard, R.W.: Interactive Learning with a “Society of Models”. Pattern Recognition 30(4), 565–581 (1997)
Müller, H., Müller, W., Squire, D.M., Marchand-Maillet, S., Pun, T.: Long-Term Learning from User Behavior in Content-Based Image Retrieval. Technical report, University of Geneva (2000)
Nakazato, M., Dagli, C., Huang, T.S.: Evaluating Group-based Relevance Feedback for Content-based Image Retrieval. In: Proc. IEEE Int’l. Conf. on Image Processing, pp. 599–602 (2003)
Nedovic, V., Marques, O.: A Collaborative, Long-term Learning Approach to Using Relevance Feedback in Content-based Image Retrieval Systems. In: Int’l Symposium ELMAR, pp. 143–146 (2005)
Niblack, W., Barber, R., Equitz, W., Flickner, M., Glasman, E., Petkovic, D., Yanker, P., Faloutsos, C., Taubin, G.: The QBIC project: Quering Images by Content using Color, Texture, and Shape. In: Proc. SPIE Storage and Retrieval for Image and Video Databases, pp. 173–187 (1993)
Ortega, M., Rui, Y., Chakrabarti, K., Mehrotra, S., Huang, T.S.: Supporting similarity queries in MARS. In: Proc. ACM Int’l Conf. Multimedia, pp. 403–413 (1997)
Pass, G., Zabih, R., Miller, J.: Comparing Images Using Color Coherence Vectors. In: Proc. ACM Int’l Conf. Multimedia, pp. 65–73 (1996)
Rocchio, J.J.: Document Retrieval System: Optimization and Evaluation. PhD dissertation, Harvard Computational Lab, Harvard University, Cambridge, MA (1996)
Rubner, Y., Tomasi, C., Guibas, L.J.: The Earth Mover’s Distance as a Metric for Image Retrieval. Int’l J. Computer Vision 40(2), 99–121 (2000)
Rui, Y., Huang, T.S., Ortega, M., Mehrotra, S.: Relevance Feedback: A Power Tool in Interactive Content-based Image Retrieval. IEEE Trans. Circuits and Systems for Video Technology 8(5), 644–655 (1998)
Sav, S., O’Connor, N., Smeaton, A., Murphy, N.: Associating Low-level Features with Semantic Concepts using Video Objects and Relevance Feedback. In: Int’l Workshop on Image Analysis for Multimedia Interactive Services (2005)
Schapire, R.E.: The Strength of Weak Learnability. Machine Learning 5(2), 197–227 (1990)
Schmid, C., Mohr, R.: Local Grayvalue Invariants for Image Retrieval. IEEE Trans. Pattern Analysis and Machine Intelligence 19(5), 530–534 (1997)
Smeulders, A.W.M., Worring, M., Santini, S., Gupta, A., Jain, R.: Content-based image retrieval at the end of the early years. IEEE Trans. Pattern Analysis and Machine Intelligence 22(12), 1,349–1,380 (2000)
Su, Z., Li, S., Zhang, H.: Extraction of Feature Subspaces for Content-Based Retrieval Using Relevance Feedback. In: Proc. of ACM Multimedia, pp. 98–106 (2001)
Su, Z., Zhang, H., Li, S., Ma, S.: Relevance Feedback in Content-Based Image Retrieval: Bayesian Framework, Feature Subspaces, and Progressive Learning. IEEE Trans. Image Processing 12(8), 924–937 (2003)
Swain, M., Ballard, D.: Color Indexing. Int’l J. Computer Vision 7(1), 11–32 (1991)
Tai, X., Ren, F., Kita, K.: Long-Term Relevance Feedback Using Simple PCA and Linear Transformation. In: Proc. Int’l Workshop on Database and Expert Systems Applications, pp. 261–268 (2002)
Tamura, H., Mori, S., Yamawaki, T.: Texture Features Corresponding to Visual Perception. IEEE Trans. Systems, Man, and Cybernetics 8(6), 460–473 (1978)
Tao, D., Tang, X., Li, X., Wu, X.: Asymmetric Bagging and Random Subspace for Support Vector Machines-based Relevance Feedback in Image Retrieval. IEEE Trans. Pattern Analysis and Machine Intelligence 28(7), 1,088–1,099 (2006)
Tao, D., Tang, X., Li, X., Rui, Y.: Kernel Direct Biased Discriminant Analysis: A New Content-based Image Retrieval Relevance Feedback Algorithm. IEEE Trans. Multimedia 8(4), 716–727 (2006)
Tao, D., Li, X., Maybank, S.J.: Negative Samples Analysis in Relevance Feedback. IEEE Trans. Knowledge and Data Engineering 19(4), 568–580 (2007)
Tong, S., Chang, E.: Support Vector Machine Active Learning for Image Retrieval. In: Proc. ACM Int’l Conf. Multimedia, pp. 107–118 (2001)
Torralba, A., Fergus, R., Freeman, W.T.: 80 Million Tiny Images: A Large Dataset for Non-parametric Object and Scene Recognition. IEEE Trans. Pattern Analysis and Machine Intelligence 30(11), 1958–1970 (2008)
Torre, F.D.L., Kanade, T.: Multimodal Oriented Discriminant Analysis. In: Int’l Conf. Machine Learning (2005)
Tversky, A.: Features of Similarity. Psychological Review 84(4), 327–352 (1977)
Urban, J., Jose, J.M.: Adaptive Image Retrieval using a Graph Model for Semantic Feature Integration. In: Proc. ACM Int’l Workshop on Multimedia Information Retrieval, pp. 117–126 (2006)
Vapnik, V.: The Nature of Statistical Learning Theory. Springer (1995)
Vasconcelos, N., Kunt, M.: Content-based Retrieval from Image Databases: Current Solutions and Future Directions. In: Proc. IEEE Int’l Conf. Image Processing, vol. 3, pp. 6–9 (2001)
Wacht, M., Shan, J., Qi, X.: A Short-Term and Long-Term Learning Approach for Content-Based Image Retrieval. In: Proc. IEEE Int’l Conf. Acoustics, Speech, and Signal Processing, vol. 2, pp. 389–392 (2006)
Wang, J.Z., Li, J., Wiederhold, G.: SIMPLIcity: Semantics-Sensitive Integrated Matching for Picture Libraries. IEEE Trans. Pattern Analysis and Machine Intelligence 23(9), 947–963 (2001)
Yang, J., Li, Q., Zhuang, Y.: Image Retrieval and Relevance Feedback using Peer Indexing. In: Proc. IEEE Int’l Conf. Multimedia and Expo, vol. 2, pp. 409–412 (2002)
Yin, P.Y., Bhanu, B., Chang, K.C., Dong, A.: Integrating Relevance Feedback Techniques for Image Retrieval Using Reinforcement Learning. IEEE Trans. Pattern Analysis and Machine Intelligence 27(10), 1,536–1,551 (2005)
Yoshizawa, T., Schweitzer, H.: Long-term Learning of Semantic Grouping from Relevance-feedback. In: Proc. ACM SIGMM Int’l Workshop on Multimedia Information Retrieval, pp. 165–172 (2004)
Zhang, L., Lin, F., Zhang, B.: Support Vector Machine Learning for Image Retrieval. In: Proc. IEEE Int. Conf. Image Processing, vol. 2, pp. 21–724 (2001)
Zhou, X.S., Huang, T.S.: Comparing Discriminating Transformations and SVM for Learning during Multimedia Retrieval. In: Proc. ACM International Conference on Multimedia (2001)
Zhou, X.S., Huang, T.S.: Relevance feedback in image retrieval: A comprehensive review. Multimedia Systems 8(6), 536–544 (2003)
Zhou, X.S., Huang, T.S.: Small Sample Learning during Multimedia Retrieval using Biasmap. In: Proc. IEEE Int’l Conf. Computer Vision and Pattern Recognition, vol. 1, pp. 11–17 (2001)
Zhou, X.D., Zhang, L., Liu, L., Zhang, Q., Shi, B.: A Relevance Feedback Method in Image Retrieval by Analyzing Feedback Log File. In: Proc. Int’l Conf. Machine Learning and Cybernetics, vol. 3, pp. 1,641–1,646 (2002)
Zhou, Z.H.: Ensemble Learning. In: Encyclopedia of Biometrics, pp. 270–273 (2009)
Zhuang, Y., Yang, J., Li, Q., Pan, Y.: A Graphic-theoretic Model for Incremental Relevance Feedback in Image Retrieval. In: Proc. Int’l Conf. Image Processing, vol. 1, pp. 413–416 (2002)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2013 Springer-Verlag Berlin Heidelberg
About this chapter
Cite this chapter
Li, J., Allinson, N.M. (2013). Relevance Feedback in Content-Based Image Retrieval: A Survey. In: Bianchini, M., Maggini, M., Jain, L. (eds) Handbook on Neural Information Processing. Intelligent Systems Reference Library, vol 49. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-36657-4_13
Download citation
DOI: https://doi.org/10.1007/978-3-642-36657-4_13
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-36656-7
Online ISBN: 978-3-642-36657-4
eBook Packages: EngineeringEngineering (R0)