skip to main content
10.1145/1743384.1743413acmconferencesArticle/Chapter ViewAbstractPublication PagesmmConference Proceedingsconference-collections
research-article

A classification-driven similarity matching framework for retrieval of biomedical images

Published: 29 March 2010 Publication History

Abstract

This paper presents a classification-driven biomedical image retrieval system to bride the semantic gap by transforming image features to their global categories at different granularity, such as image modality, body part, and orientation. To generate the feature vectors at different levels of abstraction, both the visual concept feature based on the "bag of concepts" model that comprise of local color and texture patches and various low-level global color, edge, and texture-related features are extracted. Since, it is difficult to find a unique feature to compare images effectively for all types of queries, we utilize a similarity fusion approach based on the linear combination of individual features. However, instead of using the commonly used fixed or hard weighting approach, we rely on the image classification to determine the importance of a feature at real time. For this, a supervised multi-class classifier based on the support vector machine (SVM) is trained on a set of sample images and classifier combination techniques based on the rules derived from the Bayes's theorem are explored. After the combined prediction of the classifiers for a query image category, the individual pre-computed weights of different features are adjusted in the similarity matching function for effective query-specific retrieval. Experiment is performed in a diverse medical image collection of 67,000 images of different modalities. It demonstrates the effectiveness of the category-specific similarity fusion approach with a mean average precision (MAP) score of 0.0265 when compared to using only a single feature or equal weighting of each feature in similarity matching.

References

[1]
Chang, S. F., Sikora, T., and Puri, A. 2001. Overview of the mpeg-7 standard. IEEE T. Circ. Syst. Vid. 11, 6 (Jun. 2001), 688--695.
[2]
Chang, C. C., and Lin, C. J. 2006. LIBSVM : a library for support vector machines. Software available at http://www.csie.ntu.edu.tw/cjlin/libsvm
[3]
Duan, K., and Keerthi, S. S. 2005. Which is the best multiclass SVM method? An empirical study. In Proceedings of the LNCS. 3541 Springer, Sixth International Workshop on Multiple Classifier Systems. (Seaside, CA, USA, Jun. 13-15, 2005).
[4]
Duda, R. O., Hart, P. E., and Stork, D. G. 2001 Pattern Classification. John Wiley & Sons Ltd.
[5]
Florea, F., Müller H., Rogozan, A., Geissbuhler, A., and Darmoni, S. 2006. Medical image categorization with MedIC and MedGIFT. In Proceedings of the Med Inform Europe (MIE 2006). (Maastricht, Netherlands, Jul. 27-30, 2006).
[6]
Fox, E. A., and Shaw, J. A. 1994. Combination of Multiple Searches. In Proceedings of the 2nd Text Retrieval Conference (TREC-2). NIST Special Publication. 243--252.
[7]
Grubinger, M., Clough, P., Hanbury, A., and Müller, H. 2008. Lire: lucene image retrieval: an extensible java CBIR library. In Proceedings of the 16th ACM international conference on Multimedia. (Vancouver, British Columbia, Canada, Oct. 26-31, 2008).
[8]
Haralick, R. M., Shanmugam, Dinstein. I. 1973. Textural features for image classification. IEEE T. Syst. Man Cyb. 3, (Nov. 1973), 610--621.
[9]
Hsu, W., Antani, S., Long, L. R., Neve, L., Thoma, and G. R. 2008. SPIRS: a Web-based Image RetrievalSystem For Large Biomedical Databases. Int. J. Med. Inform. (Apr. 1999), 78, 13--24.
[10]
Kittler, J., Hatef, M., Duin, R. P. W., and Matas, J. 1998. On combining Classifiers. IEEE T. Pattern. Anal. (Mar. 1998), 20, 3, 226--239.
[11]
Lee, J. H. 1995. Combining Multiple Evidence from Different Properties of Weighting Schemes. In Proceedings of the 18th annual international ACM SIGIR conference on Research and development in information retrieval. (Seattle, Washington, USA, Jul. 09-13, 1995) DOI= http://portal.acm.org/citation.cfm?id=215358.
[12]
Lehmann, T. M., Wein, B. B., Dahmen, J., Bredno, J., Vogelsang, F., and Kohnen, M. 2000. Content-based image retrieval in medical applications - A novel multi-step approach. In Proceedings of the SPIE. 3972, 312--320, Storage and Retrieval for Media Databases 2000 (San Jose, CA, USA, Jan. 26-28, 2000).
[13]
Lehmann, T. M., Güld, M. O., Deselaers, T., Keysers, D., Schubert, H., Spitzer, K., Ney, H., and Wein, B. B. 2005. Automatic categorization of medical images for content-based retrieval and data mining. Comput. Med. Imag. Grap. (Mar. 2005), 29, 2, 143--155.
[14]
Mojsilovic, A., and Gomes, J. 2002. Semantic based image categorization, browsing and retrieval in medical image databases. In Proceedings of the IEEE International Conference of Image Processing. (Rochester, New York, USA, Sep. 22-25, 2002).
[15]
Mòller, H., Michoux, N., Bandon, D., and Geissbuhler, A. 2004. A Review of Content-Based Image Retrieval Systems in Medical Applications Clinical Benefits and Future Directions. Int. J. Med. Inform. 73, 1, (Feb. 2004), 1--23.
[16]
Müller, H., Rosset, A., Vallee, J., and Geissbuhler, A. 2003. Integrating content-based visual access methods into a medical case database. In Proceedings of the Med Inform Europe (MIE 2003), (St Malo, France, May 4-7, 2003).
[17]
Müller, H., Kalpathy-Cramer, J., Kahn, Jr. C. E., Hatt, W., Bedrick, S., and Hersh, W. 2008. Overview of the ImageCLEFmed 2008 Medical Image Retrieval Task. In Proceedings of the LNCS,5706. 9th Workshop of the Cross-Language Evaluation Forum, CLEF 2008, (Aarhus, Denmark, Sep. 17-19, 2008). Revised Selected Papers 2009.
[18]
Mòller, H., Kalpathy-Cramer, Eggel, I., Bedrick, S., Radhouani, S., Bakke, B., Kahn, Jr. C. E., and Hersh, W. 2009. Overview of the CLEF 2009 Medical Image Retrieval Track. Working Notes for the CLEF 2009 Workshop, (Corfu, Greece, Sep. 30-Oct. 2, 2009).
[19]
Pass, G., Zabih, R., and Miller, J. 1997. Comparing images using color coherence vectors. In Proceedings of the fourth ACM international conference on Multimedia (Boston, Massachusetts, USA, 1997). DOI= http://portal.acm.org/citation.cfm?id=244148
[20]
Rahman, M. M., Antani, S. K., and Thoma, G. R. 2009. A Medical Image Retrieval Framework in Correlation Enhanced Visual Concept Feature Space. In Proceedings of the 22nd IEEE International Symposium on Computer-Based Medical Systems (CBMS). (Albuquerque, New Mexico, USA, Aug. 3-4, 2009)
[21]
Rahman, M. M., Desai, B. C., and Bhattacharya, P. 2008. Medical Image retrieval with probabilistic multi-class support vector machine classifiers and adaptive similarity fusion. Comput. Med. Imag. Grap. (Mar. 2008), 32, 95--108.
[22]
Smeulder, A., Worring, M., Santini, S., Gupta, A., and R. Jain. 2000. Content-Based Image Retrieval at the End of the Early Years. IEEE Trans. Pattern Anal. Machine Intell. 22, 12, (Dec. 2000), 1349--1380.
[23]
Shyu, C. R., Brodley, C. E., Kak, A. C., Kosaka, A., Aisen, A. M., and Broderick, L. S. 1999. ASSERT: a physician-in-the-loop content-based image retrieval system for HRCT image databases. Comput. Vis. Image. Und. 75, (Jul. 1999), 111--132.
[24]
Vapnik, V. 1998 Statistical Learning Theory. Wiley-Interscience.
[25]
Wu, T. F., Lin, C. J., and Weng, R. C. 2004. Probability Estimates for Multi-class Classification by Pairwise Coupling. J. Mach. Learn. Res. 5 (Aug. 2004), 975--1005.
[26]
Yates, R. B., and Neto, B.R. 1999 Modern Information Retrieval. Addison Wesley.

Cited By

View all
  • (2019)Multi-feature fusion method for medical image retrieval using wavelet and bag-of-featuresComputer Assisted Surgery10.1080/24699322.2018.156008724:sup1(72-80)Online publication date: 28-Jan-2019
  • (2018)A Survey of Techniques Used in Processing and Mining of Medical ImagesData Science and Analytics10.1007/978-981-10-8527-7_13(139-155)Online publication date: 8-Mar-2018
  • (2016)Biomedical Image Classification with Multi Response Linear Regression (MLR) as Meta-Learner Combiner and Its Effectiveness on Small to Large Data Sets2016 International Conference on Computational Science and Computational Intelligence (CSCI)10.1109/CSCI.2016.0028(110-115)Online publication date: Dec-2016
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
MIR '10: Proceedings of the international conference on Multimedia information retrieval
March 2010
600 pages
ISBN:9781605588155
DOI:10.1145/1743384
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 29 March 2010

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. classification
  2. classifier combination
  3. content-based image retrieval
  4. medical imaging
  5. similarity matching
  6. support vector machine

Qualifiers

  • Research-article

Conference

MIR '10
Sponsor:
MIR '10: International Conference on Multimedia Information Retrieval
March 29 - 31, 2010
Pennsylvania, Philadelphia, USA

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)9
  • Downloads (Last 6 weeks)5
Reflects downloads up to 21 Jan 2025

Other Metrics

Citations

Cited By

View all
  • (2019)Multi-feature fusion method for medical image retrieval using wavelet and bag-of-featuresComputer Assisted Surgery10.1080/24699322.2018.156008724:sup1(72-80)Online publication date: 28-Jan-2019
  • (2018)A Survey of Techniques Used in Processing and Mining of Medical ImagesData Science and Analytics10.1007/978-981-10-8527-7_13(139-155)Online publication date: 8-Mar-2018
  • (2016)Biomedical Image Classification with Multi Response Linear Regression (MLR) as Meta-Learner Combiner and Its Effectiveness on Small to Large Data Sets2016 International Conference on Computational Science and Computational Intelligence (CSCI)10.1109/CSCI.2016.0028(110-115)Online publication date: Dec-2016
  • (2015)Localizing global descriptors for content-based image retrievalEURASIP Journal on Advances in Signal Processing10.1186/s13634-015-0262-62015:1Online publication date: 7-Sep-2015
  • (2014)Searching images with MPEG-7 (& MPEG-7-like) Powered Localized dEscriptors: The SIMPLE answer to effective Content Based Image Retrieval2014 12th International Workshop on Content-Based Multimedia Indexing (CBMI)10.1109/CBMI.2014.6849821(1-6)Online publication date: Jun-2014
  • (2013)Content-Based and Similarity-Based Querying for Broad-Usage Medical Image RetrievalAdvances in Biomedical Infrastructure 201310.1007/978-3-642-37137-0_8(63-76)Online publication date: 2013
  • (2012)Content-Based Microscopic Image Retrieval System for Multi-Image QueriesIEEE Transactions on Information Technology in Biomedicine10.1109/TITB.2012.218582916:4(758-769)Online publication date: 1-Jul-2012
  • (2011)MedFMI-SiRProceedings of the Second international conference on Information technology in bio- and medical informatics10.5555/2035485.2035488(16-30)Online publication date: 31-Aug-2011
  • (2011)Review of medical image retrieval systems and future directionsProceedings of the 2011 24th International Symposium on Computer-Based Medical Systems10.1109/CBMS.2011.5999142(1-6)Online publication date: 27-Jun-2011

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Media

Figures

Other

Tables

Share

Share

Share this Publication link

Share on social media