Skip to main content
SpringerLink
Log in

A Review of Region-Based Image Retrieval

  • Published:
Journal of Signal Processing Systems Aims and scope Submit manuscript

Abstract

Searching interested images based on visual properties or contents of images is a challenging problem and it has received much attention from researchers in the last 20 years. The gap between low-level visual features and high-level semantic understanding of images, which is also known as the semantic gap problem, is the bottleneck to further improvement of the performance of a content-based image retrieval system. In order to solve this semantic gap problem, one of the most popular approaches in recent years is to change the focus from the global content description of images into the local content description by regions (region-based image retrieval) or even the objects in images (object-based image retrieval). Although much research in region-based image retrieval has already been done, there are still three main problems need to be tackled properly: (a) local region-based features, (b) similarity measures, and (c) relevance feedback based on regions. In this paper, we review some recent development in region-based image retrieval with respect to the above three problems and propose some future directions for region-based image retrieval research towards the end of this paper.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Figure 1
Figure 2

Similar content being viewed by others

References

  1. Goodrum, A. (2000). Image information retrieval: An overview of current research. Information Science, 3(2), 63–67.

    Google Scholar 

  2. Smeulders, A., Worring, M., Santini, S., Gupta, A., & Jain, R. (2000). Content-based image retrieval at the end of the early years. IEEE Transactions on Pattern Analysis and Machine Intelligence, 22, 1349–1380.

    Article  Google Scholar 

  3. Datta, R., Li, J., & Wang, J. (2005). Content-based image retrieval: Approaches and trends of the new age. In Proceedings of the ACM SIGMM international workshop on multimedia information retrieval (pp. 253–262).

  4. Datta, R., Joshi, D., Li, J., & Wang, J. (2008). Image retrieval: Ideas, influence, and trends of the new age. ACM Computing Surveys, 40(2), 3–63.

    Article  Google Scholar 

  5. Gevers, T., & Smeulders, A. (1997). PicToSeek: A content-based image search engine for the WWW. In Proceedings of international conference on visual information systems (pp. 93–100).

  6. Sclaroff, S., Taycher, L., & Cascia, M. (1997). ImageRover: A content-based image browser for the world wide web. In Proceedings of IEEE international workshop on content-based access of image and video libraries (pp. 2–9).

  7. Inoue, M. (2004). On the need for annotation-based image retrieval. In Proceedings of workshop on information retrieval in context.

  8. Larish, J. (1995). Kodak’s still picture exchange for print and film use. Advanced Imaging, 10(4), 38–39.

    Google Scholar 

  9. Martucci, M. (1995). Digit still marketing at presslink. Advanced Imaging, 10(4), 34–36.

    Google Scholar 

  10. Zhou, X.-S., & Huang, T. (2002). Unifying keywords and visual contents in image retrieval. IEEE Multimedia, 9(2), 23–33.

    Article  MathSciNet  Google Scholar 

  11. Niblack, C., Barber, R., Equitz, W., Flickner, M., Glasman, E., Petkovic, D., et al. (1993). The QBIC project: Querying images by content using color, texture, and shape. In Proceedings of SPIE (pp. 173–187).

  12. Smith, J. (1997). Integrated spatial and feature image systems: Retrieval, analysis, and compression. PhD Thesis, Columbia University.

  13. Ogle, V., & Stonebraker, M. (1995). Chabot: Retrieval from a relational database of images. IEEE Computer, 28(9), 40–48.

    Google Scholar 

  14. Smith, J. R., & Chang, S.-F. (1996). VisualSEEk: A fully automated content-based image query system. In Proceedings of ACM international conference on multimedia (pp. 87–98).

  15. Huang, J., & Kumar, R. (1997). Image indexing using color correlograms. In Proceedings of IEEE conference on computer vision and pattern recognition (pp. 762–768).

  16. Wu, K., & Yap, K.-H. (2006). Fuzzy SVM for content-based image retrieval—a pseudo-label support vector machine framework. IEEE Computational Intelligence Magazine, 1, 10–16.

    Google Scholar 

  17. Ma, W.-Y., & Manjunath, B. (1997). NETRA: A toolbox for navigating large image databases. In Proceedings of IEEE international conference on image processing (pp. 568–571).

  18. Carson, C., Thomas, C., Belongie, S., Hellerstein, J., & Malik, J. (1999). Blobworld: A system for region-based image indexing and retrieval. Lecture Notes in Computer Science, 1614, 660.

    Google Scholar 

  19. Wang, J.-Z., Li, J., & Wiederhold, G. (2001). SIMPLicity: Semantics sensitive integrated matching for picture libraries. IEEE Transactions on Pattern Analysis and Machine Intelligence, 23, 947–963.

    Article  Google Scholar 

  20. Rui, Y., Huang, T., Ortega, M., & Mehrotra, S. (1998). Relevance feedback: A power tool for interactive content-based image retrieval. IEEE Transactions on Circuits and Systems for Video Technology, 6, 644–655.

    Google Scholar 

  21. Frederix, G., Caenen, G., & Pauwels, E. (2000). Panormic, adaptive and reconfigurable interface for similarity search. In Proceeding of IEEE international conference on image processing (Vol. 3, pp. 222–225).

  22. Hiroike, A., Musha, Y., Sugimoto, A., & Mori, Y. (1999). Visualization of information spaces to retrieve and browse image data. In Proceeding of visual: Information and information system (Vol. 1, pp. 152–162).

  23. Wu, K., & Yap, K.-H. (2007). Content-based image retrieval using fuzzy perceptual feedback. Multimedia Tools and Applications, 43, 235–251.

    Article  Google Scholar 

  24. Yap, K.-H., & Wu, K. (2005). A soft relevance framework in content-based image retrieval systems. IEEE Transaction on Circuits and Systems for Video Technology, 15, 1557–1568.

    Article  Google Scholar 

  25. Wang, L., Chan, K.-L., & Xue, P. (2005). A criterion for optimizing kernel parameters in KBDA for image retrieval. IEEE Transaction on Systems, Man, and Cybernetics. Part B. Cybernetics, 35, 556–562.

    Article  Google Scholar 

  26. Zhang, Y.-H., Wang, L., Hartley, R., & Li, H.-D. (2007). Where is the weet-bix. In Proceeding of the Asian conference on computer vision (pp. 800–810).

  27. Tsai, C.-F., McGarry, K., & Tait, J. (2003). Image classification using hybrid neural network. In Proceedings of the ACM SIGIR conference on research and development in information retrieval (pp. 431–432).

  28. Fan, J.-P., Gao, Y.-L., Luo, H.-Z., & Xu, G.-Y. (2004). Automatic image annotation by using concept-sensitive salient objects for image content representation. In Proceedings of the ACM SIGIR conference on research and development in information retrieval (pp. 361–368).

  29. Comanicu, D., & Meer, P. (2002). Mean shift: a robust approach toward feature space analysis. IEEE Transactions on Pattern Analysis and Machine Intelligence, 24, 603–619.

    Article  Google Scholar 

  30. Marques, O., Mayron, L., Borba, G., & Gamba, H. (2006). Using visual attention to extract regions of interest in the context of image retrieval. In Proceedings of ACM annual southeast regional conference (pp. 638–643).

  31. Wang, X.-Y., Hu, F.-L., & Yang, H.-Y. (2006). A novel regions-of-interest based image retrieval using multiple features. In Proceedings of the multi-media modelling international conference (Vol. 1, pp. 377–380).

  32. Zhang, M.-X., Lu, Z.-G., & Shen, J.-Y. (2007). Image retrieval with simple invariant features based hierarchical uniform segmentation. In Proceedings of the international conference on computational intelligence and security (Vol. 1, pp. 461–465).

  33. Deng, Y., Manjunath, B., & Shin, H. (1999). Color image segmentation. In Proceedings of IEEE conference on computer vision and pattern recognition (Vol. 2, pp. 446–451).

  34. Almeida, J., Rocha, A., Torres, R., & Goldenstein, S. (2008). Making colors worth more than a thousand words. In Proceedings of the ACM symposium on applied computing (pp. 1180–1186).

  35. MPEG-7 Overview (version 10). (2004). Online Source – http://www.chiariglione.org/mpeg/standards/mpeg-7/mpeg-7.htm#E12E25.

  36. Manjunath, B., Salembier, P., & Sikora, T. (2001). Introduction to MPEG-7: Multimedia content description interface. New York: Wiley.

    Google Scholar 

  37. Chen, Y., & Wang, J. (2002). A region-based fuzzy feature matching approach to content-based image retrieval. IEEE Transactions on Pattern Analysis and Machine Intelligence, 24, 1252–1267.

    Article  Google Scholar 

  38. Duda, R., Hart, P., & Stork, D. (2001). Pattern classification second edition. New York: Wiley.

    Google Scholar 

  39. Sun, Y.-Q., & Ozawa, S. (2004). A hierarchical approach for region-based image retrieval. In Proceedings of the IEEE conference on computer vision and pattern recognition (Vol. 2, pp. 996–1001).

  40. Jing, F., Li, M., Zhang, H., & Zhang, B. (2002). An efficient and effective region-based image retrieval framework. IEEE Transactions on Image Processing, 13, 699–709.

    Article  Google Scholar 

  41. Jing, F., Zhang, B., Lin, F.-Z., Ma, W.-Y., & Zhang, H.-J. (2001). A novel region-based image retrieval method using relevance feedback. In Proceedings of ACM workshops on multimedia: Multimedia information retrieval (pp. 28–31).

  42. Jing, F., Li, M.-J., Zhang, H.-J., & Zhang, B. (2004). An efficient and effective region-based image retrieval framework. IEEE Transactions on Image Processing, 13(5), 699–709.

    Article  Google Scholar 

  43. Xiong, W., Qiu, B., Tian, Q., Xu, C.-S., Ong, S., Foong, K., et al. (2005). MultiPRE: A novel framework with multiple parallel retrieval engines for content-based image retrieval. In Proceedings of the ACM international conference on multimedia (pp. 1023–1032).

  44. Rahmani, R., Goldman, S., Zhang, H., Krettek, J., & Fritts, J. (2005). Localized content based image retrieval. In Proceedings of the ACM SIGMM international workshop on multimedia information retrieval (pp. 227–236).

  45. Hastie, T., Tibshirani, R., & Friedman, J. (2001). The elements of statistical learning: Data mining, inference, and prediction. New York: Springer.

    MATH  Google Scholar 

  46. Zhang, R.-F., & Zhang, Z.-F. (2004). Hidden semantic concept discovery in region based image retrieval. In Proceedings of the IEEE international conference on systems, man and cybernetics (Vol. 1, pp. 1117–1124).

  47. He, X. (2004). Incremental semi-supervised subspace learning for image retrieval. In Proceedings of the ACM conference on multimedia (Vol. 1, pp. 2–8).

  48. Lin, Y., Liu, T., & Chen, H. (2005). Semantic manifold learning for image retrieval. In Proceedings of the ACM conference on multimedia (Vol. 1, pp. 249–258).

  49. Yu, J., & Tian, Q. (2006). Learning image manifolds by semantic subspace projection. In Proceedings of the ACM conference on multimedia (Vol. 1, pp. 297–306).

  50. Cai, D., He, X., & Han, J. (2007). Spectral regression: A unified subspace learning framework for content-based image retrieval. In Proceedings of the ACM conference on multimedia (Vol. 3, pp. 403–412).

  51. Greenspan, H., Dvir, G., & Rubner, Y. (2000). Region correspondence for image matching via EMD flow. In Workshop on content-based access of image and video libraries (pp. 27–31).

  52. Li, J., Wang, J., & Wiederhold, G. (2000). IRM: Integrated region matching for image retrieval. In Proceedings of the ACM international conference on multimedia (pp. 147–156).

  53. Wang, J., & Du, Y.-P. (2001). Scalable integrated region-based image retrieval using IRM and statistical clustering. In Proceedings of the ACM/IEEE-CS joint conference on digital libraries (pp. 268–277).

  54. Wang, J. (2000). Region-based retrieval of biomedical images. In Proceedings of the ACM international conference on multimedia (pp. 511–512).

  55. Chen, Y.-X., & Wang, J. (2003). Looking beyond region boundaries: A robust image similarity measure using fuzzified region features. In Proceedings of the IEEE international conference on fuzzy systems (Vol. 2, pp. 1165–1170).

  56. Weber, R., & Mlivoncic, M. (2003). Efficient region-based image retrieval. In Proceedings of the international conference on information and knowledge management (pp. 69–76).

  57. Sun, Y.-Q., & Ozawa, S. (2003). Semantic-meaningful content-based image retrieval in wavelet domain. In Proceedings of ACM SIGMM international workshop on multimedia information retrieval (pp. 122–129).

  58. Hsieh, J., & Grimson, W. (2003). Spatial template extraction for image retrieval by region matching. IEEE Transactions on Image Processing, 12(11), 1404–1415.

    Article  Google Scholar 

  59. Lv, Q., Charikar, M., & Li, K. (2004). Image similarity search with compact data structures. In Proceedings of the ACM international conference on information and knowledge management (pp. 208–217).

  60. Vasconcelos, N. (2000). Bayesian models for visual information retrieval. Cambridge: Massachusetts Institute of Technology.

    Google Scholar 

  61. Guo, G., Jain, A., Ma, W.-Y., Zhang, H.-J. (2002). Learning similarity measure for natural image retrieval with relevance feedback. IEEE Transactions on Neural Network, 13, 811–820.

    Article  Google Scholar 

  62. Hertz, T., Bar-Hillel, A., & Weinshall, D. (2004). Learning distance functions for image retrieval. In Proceedings of IEEE conference on computer vision and pattern recognition (Vol. 2, pp. 570–577).

  63. Si, L., Jin, R., Hoi, S., & Lyu, M (2006). Collaborative image retrieval via regularized metric learning. Multimedia Systems, 12, 34–44.

    Article  Google Scholar 

  64. Globerson, A., & Roweis, S. (2005). Metric learning by collapsing classes. In Proceedings of advances in neural information processing systems.

  65. Weinberger, K., Blitzer, J., & Saul, L. (2006). Distance metric learning for large margin nearest neighbor classification. In Proceedings of advances in neural information processing systems (pp. 1473–1480).

  66. Chen, Y., Wang, J., & Krovetz, R. (2005). Clue: Cluster-based retrieval of images by unsupervised learning. IEEE Transactions on Image Processing, 14, 1187–1201.

    Article  Google Scholar 

  67. Chen, Y., Wang, J., & Krovetz, R. (2003). An unsupervised learning approach to content-based image retrieval. In Proceedings of IEEE international symposium of signal processing and applications (Vol. 1, pp. 197–200).

  68. Chapelle, O., Scholkopf, B., & Zien, A. (2006). Semi-supervised learning. Cambridge: MIT.

    Google Scholar 

  69. Wagsta, K., & Cardie, C. (2000). Clustering with instance-level constraints. In Proceedings of the IEEE international conference on machine learning (Vol. 1, pp. 1103–1110).

  70. Wagsta, K., Cardie, C., Rogers, S., & Schroell, S. (2001). Constrained k-means clustering with background knowledge. In Proceedings of the IEEE international conference on machine learning (Vol. 1, pp. 577–584).

  71. Klein, D., & Kamvar, S. (2002). From instance-level constraints to space-level constraints: Making the most of prior knowledge in data clustering. In Proceedings of the IEEE international conference on machine learning (Vol. 1, pp. 307–314).

  72. Xing, E., Ng, A., Jordan, M., & Russell, S. (2003). Distance metric learning, with application to clustering with side-information. In Proceedings of the advances in neural information processing systems (Vol. 1, pp. 505–512).

  73. Bar-Hillel, A., Hertz, T., Shental, N., & Weinshall, D. (2003). Learning distance functions using equivalence relations. In Proceedings of IEEE international conference on machine learning (Vol. 1, pp. 11–18).

  74. Shental, N., Bar-Hillel, A., Hertz, T., & Weinshall, D. (2004). Computing Gaussian mixture models with EM using side-information. In Proceedings of the advances in neural information processing systems.

  75. Chang, H., & Yeung, D.-Y. (2005). Stepwise metric adaptation based on semi-supervised learning for boosting image retrieval performance. In Proceedings of the British machine vision conference.

  76. Heisterkamp, D., Peng, J., & Dai, H.-K. (2001). Adaptive quasiconformal kernel metric for image retrieval. In Proceedings of IEEE conference on computer vision and pattern recognition (Vol. 2, pp. 388–393).

  77. Chang, H., & Yeung, D. (2006). Locally linear metric adaption with application to semi-supervised clustering and image retrieval. Pattern Recognition, 39, 1253–1264.

    Article  MATH  Google Scholar 

  78. Frome, A., Singer, Y., & Malik, J. (2006). Image retrieval and classification using local distance functions. In Proceedings of advances in neural information processing systems.

  79. Ulusoy, I., & Bishop, C. (2005). Generative versus discriminative methods for object recognition. In Proceedings of IEEE conference on computer vision and pattern recognition.

  80. Tao, D., Tang, X., Li, X., & Wu, X. (2006). Asymmetric bagging and random subspace for support vector machines- based relevance feedback in image retrieval. IEEE Transactions on Pattern Analysis and Machine Intelligence, 28(7), 1088–1099.

    Article  Google Scholar 

  81. Wang, L., Chan, K.-L., & Zhang, Z.-H. (2003). Bootstrapping SVM active learning by incorporating unlabelled images for image retrieval. In Proceedings of IEEE conference on computer vision and pattern recognition.

  82. Tieu, K., & Viola, P. (2004). Boosting image retrieval. International Journal of Computer Vision, 56(1-2), 17–36.

    Article  Google Scholar 

  83. Jiang, W., Er, G., Dai, Q., & Gu, J. (2006). Similarity-based online feature selection in content-based image retrieval. IEEE Transactions on Image Processing, 15(3), 702–712.

    Article  Google Scholar 

  84. Tong, S., & Chang, E. (2001). Support vector machine active learning for image retrieval. In Proceedings of ACM international conference on multimedia (pp. 107–118).

  85. Westerveld, T., & De Vries, A. (2005). Experimental evaluation of a generative probabilistic image retrieval model on ’easy’ data. In Proceedings of multimedia information retrieval workshop.

  86. Jiang, W., Chan, K.-L., Li, M.-J., & Zhang, H.-J. (2005). Mapping low-level features to high-level semantic concepts in region-based image retrieval. In Proceedings of IEEE conference on computer vision and pattern recognition (Vol. 2, pp. 244–249).

  87. Crisianini, N., & Shawe-Taylor, J. (2000). An introduction to support vector machines and other kernel-based learning methods. Cambridge University Press.

  88. Zhang, L., Lin, F., & Zhang, B. (2001). Support vector machine learning for image retrieval. In Proceedings of IEEE international conference on image processing (pp. 721–724).

  89. Chen, Y.-Q., Zhou, X., & Huang, T. (2001). One-class SVM for learning in image retrieval. In Proceedings of IEEE international conference on image processing.

  90. Wang, L., Gao, Y., Chan, K.-L., Xue, P., & Yau, W.-Y. (2005). Retrieval with knowledge-driven kernel design: an approach to improving SVM-based CBIR with relevance. In Proceedings of IEEE international conference on computer vision.

  91. Viola, P., & Jones, M. (2001). Rapid object detection using a boosted cascade of simple features. In Proceedings of IEEE conference on computer vision and pattern recognition (pp. 511–518).

  92. Amores, J., Sebe, N., Radeva, P., Gevers, T., & Smeulders, A. (2004). Boosting contextual information in content-based image retrieval. In Proceedings of ACM SIGMM international workshop on multimedia information retrieval (pp. 31–38).

  93. Schapire, R. (2003). The boosting approach to machine learning: An overview. New York: Springer.

    Google Scholar 

  94. Nguyen, G., & Worring, M. (2003). Query definition using interactive saliency. In Proceedings of the ACM SIGMM international workshop on multimedia information retrieval (pp. 150–156).

  95. Silva, S., Barcelos, C., & Batista, M. (2006). An image retrieval system adaptable to user’s interests by the use of relevance feedback via genetic algorithm. In Proceedings of the Brazilian symposium on multimedia and web (pp. 45–52).

  96. Guan, J., & Qiu, G.-P. (2007). Learning user intention in relevance feedback using optimization. In Proceedings of the ACM SIGMM international workshop on multimedia information retrieval (pp. 41–50).

  97. Savova, V., Tenenbaum, J., Kaelbling, L., & Yuille, A. (2007). The grammar of vision: Probabilistic grammar-based models for visual scene understanding and object categorization. In Proceedings of the annual conference on neural information processing systems.

Download references

Author information

Authors and Affiliations

  1. School of Electrical and Electronic Engineering, Nanyang Technological University, Singapore, 639798, Singapore

    Wei Huang, Yan Gao & Kap Luk Chan

Authors
  1. Wei Huang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yan Gao
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Kap Luk Chan
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Wei Huang.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Huang, W., Gao, Y. & Chan, K.L. A Review of Region-Based Image Retrieval. J Sign Process Syst Sign Image Video Technol 59, 143–161 (2010). https://doi.org/10.1007/s11265-008-0294-3

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11265-008-0294-3

Keywords

Search

Navigation