Abstract
In this paper we propose a system for medical image retrieval using multimodal data. The system can be separated in an off-line and on-line phase. The off-line phase deals with modality classification of the images by their visual content. For this part we use state-of-the-art opponentSIFT visual features to describe the image content, as for the classification we use SVMs. The modality classification labels all images in the database with their corresponding modality. The off-line phase, also, implements the text-based retrieval structure of the system. In this part we index the text associated with the images using the open-source search engine Terrier IR. In the on-line phase the retrieval is performed. In this phase the system receives a text query. The system processes the query and performs the text-based retrieval with Terrier IR and the initial results are generated. Afterwards, the images in the initial results are re-ranked based on their modality and the final results are provided. Our system was evaluated against the standardized ImageCLEF 2013 medical dataset. Our system reported results with a mean average precision of 0.32, which is state-of-the-art performance on the dataset.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Choplin, R., Boehme, J., Maynard, C.: Picture archiving and communication systems: an overview. Radiographics 12(1), 127–129 (1992)
de Herrera, A.G.S., Kalpathy-Cramer, J., Fushman, D.D., Antani, S., Müller, H.: Overview of the imageclef 2013 medical tasks. In: Working notes of CLEF 2013 (2013)
Lehmann, T.M., Wein, B.B., Dahmen, J., Bredno, J., Vogelsang, F., Kohnen, M.: Content-based image retrieval in medical applications: a novel multistep approach. In: Proceedings of SPIE: Storage and Retrieval for Media Databases, vol. 3972, pp. 312–320 (2000)
Shyu, C.-R., Brodley, C.E., Kak, A.C., Kosaka, A., Aisen, A.M., Broderick, L.S.: Assert: A physician-in-the-loop content-based retrieval system for HRCT image databases. Computer Vision and Image Understanding 75(12), 111–132 (1999)
Simonyan, K., Modat, M., Ourselin, S., Cash, D., Criminisi, A., Zisserman, A.: Immediate ROI search for 3-D medical images. In: Greenspan, H., Müller, H., Syeda-Mahmood, T. (eds.) MCBR-CDS 2012. LNCS, vol. 7723, pp. 56–67. Springer, Heidelberg (2013)
El-Naqa, I., Yang, Y., Galatsanos, N.P., Nishikawa, R.M., Wernick, M.N.: A similarity learning approach to content-based image retrieval: application to digital mammography. IEEE Transactions on Medical Imaging 23(10), 1233–1244 (2004)
Müller, H., de Herrera, A.G.S., Kalpathy-Cramer, J., Demner-Fushman, D., Antani, S., Eggel, I.: Overview of the imageclef 2012 medical image retrieval and classification tasks. In: CLEF (Online Working Notes/Labs/Workshop) (2012)
Medical retrieval task, http://www.imageclef.org/node/104/ (accessed: July 03, 2014)
Guld, M.O., Kohnen, M., Keysers, D., Schubert, H., Wein, B.B., Bredno, J., Lehmann, T.M.: Quality of DICOM header information for image categorization. In: Medical Imaging 2002: PACS and Integrated Medical Information Systems: Design and Evaluation, SPIE, vol. 4685, pp. 280–287 (2002)
van de Sande, K., Gevers, T., Snoek, C.: Evaluating color fescriptors for object and scene recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence 32(9), 1582–1596 (2010)
Hearst, M.A., Divoli, A., Guturu, H., Ksikes, A., Nakov, P., Wooldridge, M.A., Ye, J.: Biotext search engine: beyond abstract search. Bioinformatics 23(16), 2196–2197 (2007)
Kyriakopoulou, A., Stathopoulos, S., Lourentzou, I., Kalamboukis, T.: Ipl at clef 2013 medical retrieval task. In: CLEF (Online Working Notes/Labs/Workshop) (2013)
Kahn Jr., C.E., Thao, C.: Goldminer: a radiology image search engine. American Journal of Roentgenology 188(6), 1475–1478 (2007)
Xu, S., McCusker, J., Krauthammer, M.: Yale image finder (yif): a new search engine for retrieving biomedical images. Bioinformatics 24(17), 1968–1970 (2008)
Ceylan, N.M., Ozturkmenoglu, O., Alpkocak, A.: Demir at imageclefmed 2013: The effects of modality classification to information retrieval. In: CLEF (Online Working Notes/Labs/Workshop) (2013)
Rahman, M.M., You, D., Simpson, M.S., Antani, S.K., Demner-Fushman, D., Thoma, G.R.: Multimodal biomedical image retrieval using hierarchical classification and modality fusion. International Journal of Multimedia Information Retrieval 2(3), 159–173 (2013)
Kitanovski, I., Trojacanec, K., Dimitrovski, I., Loshkovska, S.: Merging words and concepts for medical articles retrieval. In: Proceedings of the 10th Conference on Open Research Areas in Information Retrieval, pp. 25–28. Le Centre De Hautes Etudes Internationales D’Informatique Documentaire (2013)
Kitanovski, I., Trojacanec, K., Dimitrovski, I., Loskovska, S.: Multimodal medical image retrieval. In: Markovski, S., Gushev, M. (eds.) ICT Innovations 2012. AISC, vol. 207, pp. 81–89. Springer, Heidelberg (2013)
Ounis, I., Amati, G., Plachouras, V., He, B., Macdonald, C., Johnson, D.: Terrier information retrieval platform. In: Losada, D.E., Fernández-Luna, J.M. (eds.) ECIR 2005. LNCS, vol. 3408, pp. 517–519. Springer, Heidelberg (2005)
Macdonald, C., Plachouras, V., He, B., Lioma, C., Ounis, I.: University of Glasgow at webclef 2005: Experiments in per-field normalisation and language specific stemming. In: Peters, C., Gey, F.C., Gonzalo, J., Müller, H., Jones, G.J.F., Kluck, M., Magnini, B., de Rijke, M., Giampiccolo, D., et al. (eds.) CLEF 2005. LNCS, vol. 4022, pp. 898–907. Springer, Heidelberg (2006)
Amati, G., Van Rijsbergen, C.J.: Probabilistic models of information retrieval based on measuring the divergence from randomness. ACM Transactions on Information Systems (TOIS) 20(4), 357–389 (2002)
Kitanovski, I., Dimitrovski, I., Loskovska, S.: Fcse at medical tasks of imageclef 2013. In: CLEF (Online Working Notes/Labs/Workshop) (2013)
Dimitrovski, I., Kocev, D., Loskovska, S., Dzeroski, S.: Hierarchical annotation of medical images. Pattern Recognition 44(10-11), 2436–2449 (2011)
Tommasi, T., Orabona, F., Caputo, B.: Discriminative cue integration for medical image annotation. Pattern Recognition Letters 29(15), 1996–2002 (2008)
Lowe, D.G.: Distinctive image features from scale-invariant keypoints. International Journal of Computer Vision 60(2), 91–110 (2004)
van Gemert, J.C., Veenman, C.J., Smeulders, A.W.M., Geusebroek, J.M.: Visual word ambiguity. IEEE Transactions on Pattern Analysis and Machine Intelligence 99(1)
Chang, C.-C., Lin, C.-J.: LIBSVM: a library for support vector machines (2001), software available at http://www.csie.ntu.edu.tw/~cjlin/libsvm
Lin, H.-T., Lin, C.-J., Weng, R.C.: A note on Platt’s probabilistic outputs for support vector machines. Machine Learning 68, 267–276 (2007)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2014 Springer International Publishing Switzerland
About this paper
Cite this paper
Kitanovski, I., Dimitrovski, I., Madjarov, G., Loskovska, S. (2014). Medical Image Retrieval Using Multimodal Data. In: Džeroski, S., Panov, P., Kocev, D., Todorovski, L. (eds) Discovery Science. DS 2014. Lecture Notes in Computer Science(), vol 8777. Springer, Cham. https://doi.org/10.1007/978-3-319-11812-3_13
Download citation
DOI: https://doi.org/10.1007/978-3-319-11812-3_13
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-11811-6
Online ISBN: 978-3-319-11812-3
eBook Packages: Computer ScienceComputer Science (R0)