Abstract
Content-based medical image retrieval has been proposed as a technique that allows not only for easy access to images from the relevant literature and electronic health records but also for training physicians, for research and clinical decision support. The bag-of-visual-words approach is a widely used technique that tries to shorten the semantic gap by learning meaningful features from the dataset and describing documents and images in terms of the histogram of these features. Visual vocabularies are often redundant, over-complete and noisy. Larger than required vocabularies lead to high-dimensional feature spaces, which present important disadvantages with the curse of dimensionality and computational cost being the most obvious ones. In this article a visual vocabulary pruning and descriptor transformation technique is presented. It enormously reduces the amount of required words to describe a medical image dataset with no significant effect on the accuracy. Results show that a reduction of up to 90 % can be achieved without impact on the system performance. Obtaining a more compact representation of a document enables multimodal description as well as using classifiers requiring low-dimensional representations.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
- 1.
- 2.
- 3.
- 4.
CIELab is a color space defined by the International Commission on Illumination (Commission Internationale de l’Éclairage) describing all colors visible for humans while trying to mimic the nonlinear response of the eye.
References
Müller, H., Michoux, N., Bandon, D., & Geissbuhler, A. (2004). A review of content-based image retrieval systems in medicine-clinical benefits and future directions. International Journal of Medical Informatics, 73(1), 1–23.
Akgül, C., Rubin, D., Napel, S., Beaulieu, C., Greenspan, H., & Acar, B. (2011). Content-based image retrieval in radiology: Current status and future directions. Journal of Digital Imaging, 24(2), 208–222.
Tang, L. H. Y., Hanka, R., & Ip, H. H. S. (1999). A review of intelligent content-based indexing and browsing of medical images. Health Informatics Journal, 5, 40–49.
Demner-Fushman, D., Antani, S., Siadat, M.-R., Soltanian-Zadeh, H., Fotouhi, F., & Elisevich, K. (2007). Automatically finding images for clinical decision support. In Proceedings of the Seventh IEEE International Conference on Data Mining Workshops, ICDMW ’07 (pp. 139–144). Washington, DC: IEEE Computer Society.
Caputo, B., Müller, H., Mahmood, T. S., Kalpathy-Cramer, J., Wang, F., & Duncan, J. (2009). Editorial of miccai workshop proceedings on medical content-based retrieval for clinical decision support. In Lecture Notes in Computer Science: Vol. 5853. Proceedings on MICCAI Workshop on Medical Content-Based Retrieval for Clinical Decision Support. Heidelberg: Springer.
Müller, H., Kalpathy-Cramer, J., Kahn, Jr. C. E., & Hersh, W. (2009). Comparing the quality of accessing the medical literature using content-based visual and textual information retrieval. In SPIE Medical Imaging, Orlando, FL (Vol. 7264, pp. 1–11).
Deserno, T. M., Antani, S., & Long, L. R. (2009). Content-based image retrieval for scientific literature access. Methods of Information in Medicine, 48(4), 371–380.
Müller, H., de Herrera, A. G. S., Kalpathy-Cramer, J., Fushman, D. D., Antani, S., & Eggel, I. (2012). Overview of the ImageCLEF 2012 medical image retrieval and classification tasks. In Working Notes of CLEF 2012 (Cross Language Evaluation Forum).
Müller, H., Clough, P., Deselaers, T., & Caputo, B., (Eds.). (2010). ImageCLEF: Experimental evaluation in visual information retrieval. The Springer International Series on Information Retrieval (Vol. 32). Berlin/Heidelberg: Springer.
Leibe, B., & Grauman, K. (2011). Visual object recognition. San Rafael, CA: Morgan & Claypool Publishers.
Foncubierta-Rodríguez, A., Depeursinge, A., & Müller, H. (2012). Using multiscale visual words for lung texture classification and retrieval. In H. Greenspan, H. Müller, & T. S. Mahmood, (Eds.), Lecture Notes in Computer Sciences: Vol. 7075. Medical content-based retrieval for clinical decision support (pp. 69–79) MCBR-CDS 2011.
Hinneburg, A., & Gabriel, H.-H. (2007). DENCLUE 2.0: Fast clustering based on kernel density estimation. Advances in Intelligent Data Analysis VII, 4723/2007, 70–80.
Haas, S., Donner, R., Burner, A., Holzer, M., & Langs, G. (2011). Superpixel-based interest points for effective bags of visual words medical image retrieval. In H. Greenspan, H. Müller & T. Syeda-Mahmood (Eds.), Lecture Notes in Computer Sciences: Vol. 7075. Medical content-based retrieval for clinical decision support, MCBR-CDS 2011.
Avni, U., Greenspan, H., Konen, E., Sharon, M., & Goldberger, J. (2011). X-ray categorization and retrieval on the organ and pathology level, using patch-based visual words. IEEE Transactions on Medical Imaging, 30(3), 733–746.
Markonis, D., de Herrera, A. G. S., Eggel, I., & Müller, H. (2012). Multi-scale visual words for hierarchical medical image categorization. In SPIE Medical Imaging 2012: Advanced PACS-Based Imaging Informatics and Therapeutic Applications (Vol. 8319, pp. 83190F–11).
Basu, S., Banerjee, A., & Mooney, R. (2002). Semi-supervised clustering by seeding. In 19th Internaional Conference on Machine Learning (ICML-2002) (pp. 19–26).
Bilenko, M., Basu, S., & Mooney, R. (2004). Integrating constraints and metric larning in semi-supervised clustering. In 21st Internaional Conference on Machine Learning (ICML-2004).
Markonis, D., Holzer, M., Dungs, S., Vargas, A., Langs, G., Kriewel, S., et al. (2012). A survey on visual information search behavior and requirements of radiologists. Methods of Information in Medicine, 51(6), 539–548.
Müller, H., Kalpathy-Cramer, J., Demner-Fushman, D., & Antani, S. (2012). Creating a classification of image types in the medical literature for visual categorization. In SPIE Medical Imaging.
Lowe, D. G. (2004). Distinctive image features from scale-invariant keypoints. International Journal of Computer Vision, 60(2), 91–110.
Yang, Y., & Newsam, S. (2010). Bag-of-visual-words and spatial extensions for land-use classification. In Proceedings of the 18th SIGSPATIAL International Conference on Advances in Geographic Information Systems, GIS ’10 (pp. 270–279). New York, NY: ACM.
Ke, Y., & Sukthankar, R. (2004). Pca-sift: A more distinctive representation for local image descriptors. In IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR 2004), Washington, DC. (Vol. 2, pp. 506–513).
Wengert, C., Douze, M., & Jégou, H. (2011). Bag-of-colors for improved image search. In Proceedings of the 19th ACM International Conference on Multimedia, MM ’11 (pp. 1437–1440). New York, NY: ACM.
Banu, M. S., & Nallaperumal, K. (2010). Analysis of color feature extraction techniques for pathology image retrieval system. IEEE.
Tirilly, P., Claveau, V., & Gros, P. (2008). Language modeling for bag-of-visual words image categorization. In Proceedings of the 2008 International Conference on Content-Based Image and Video Retrieval (pp. 249–258). New York: ACM.
Tian, Q., Zhang, S., Zhou, W., Ji, R., Ni, B., & Sebe, N. (2011). Building descriptive and discriminative visual codebook for large-scale image applications. Multimedia Tools and Applications, 51(2), 441–477.
Deerwester, S., Dumais, S. T., Furnas, G. W., Landauer, T. K., & Harshman, R. (1990). Indexing by latent semantic analysis. Journal of the American Society for Information Science, 41(6), 391–407.
Hofmann, T. (2001). Unsupervised learning by probabilistic latent semantic analysis. Machine Learning, 42(1–2), 177–196.
Bosch, A., Zisserman, A., & Munoz, X. (2006). Scene classification via plsa. In Computer Vision-ECCV 2006 (pp 517–530). Heidelberg: Springer.
Elsayad, I., Martinet, J., Urruty, T, & Djeraba, C. (2012). Toward a higher-level visual representation for content-based image retrieval. Multimedia Tools and Applications, 60(2), 455–482.
Fox, E. A., & Shaw, J. A. (1993). Combination of multiple searches. In Text Retrieval Conference (pp. 243–252).
Hand, D. J., Mannila, H., & Smyth, P. (2001). Principles of data mining (adaptive computation and machine learning). Cambridge: The MIT Press.
McG, D., Squire, Müller, H., & Müller, W. (1999). Improving response time by search pruning in a content-based image retrieval system, using inverted file techniques. In IEEE Workshop on Content-Based Access of Image and Video Libraries (CBAIVL ’99) (pp. 45–49).
Acknowledgements
This work was partially supported by the Swiss National Science Foundation (FNS) in the MANY2 project (205320-141300), the EU 7th Framework Program under grant agreements 257528 (KHRESMOI) and 258191 (PROMISE).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2015 Springer International Publishing Switzerland
About this chapter
Cite this chapter
Rodríguez, A.F., de Herrera, A.G.S., Müller, H. (2015). Meaningful Bags of Words for Medical Image Classification and Retrieval. In: Briassouli, A., Benois-Pineau, J., Hauptmann, A. (eds) Health Monitoring and Personalized Feedback using Multimedia Data. Springer, Cham. https://doi.org/10.1007/978-3-319-17963-6_5
Download citation
DOI: https://doi.org/10.1007/978-3-319-17963-6_5
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-17962-9
Online ISBN: 978-3-319-17963-6
eBook Packages: Computer ScienceComputer Science (R0)