Mandikal Vikram
ABSTRACT
Modern medical practices are increasingly dependent on Medical Imaging for clinical analysis and diagnoses of patient illnesses. A significant challenge when dealing with the extensively available medical data is that it often consists of heterogeneous modalities. Existing works in the field of Content based medical image retrieval (CBMIR) have several limitations as they focus mainly on visual or textual features for retrieval. Given the unique manifold of medical data, we seek to leverage both the visual and textual modalities to improve the image retrieval. We propose a Latent Dirichlet Allocation (LDA) based technique for encoding the visual features and show that these features effectively model the medical images. We explore early fusion and late fusion techniques to combine these visual features with the textual features. The proposed late fusion technique achieved a higher mAP than the state-of-the-art on the ImageCLEF 2009 dataset, underscoring its suitability for effective multimodal medical image retrieval.
- David M Blei, Andrew Y Ng, and Michael I Jordan. 2003. Latent dirichlet allocation. Journal of machine Learning research 3, Jan (2003), 993--1022. Google Scholar
Digital Library
- Yu Cao, Shawn Steffey, Jianbiao He, Degui Xiao, Cui Tao, Ping Chen, and Henning Müller. 2014. Medical image retrieval: a multimodal approach. Cancer informatics 13 (2014), CIN-S14053.Google Scholar
- Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. 2009. Imagenet: A large-scale hierarchical image database. In Computer Vision and Pattern Recognition, 2009. CVPR 2009. IEEE Conference on. Ieee, 248--255.Google ScholarCross Ref
- Hayit Greenspan and Adi T Pinhas. 2007. Medical image categorization and retrieval for PACS using the GMM-KL framework. IEEE Transactions on Information Technology in Biomedicine 11, 2 (2007). Google ScholarDigital Library
- Thomas Hofmann. 1999. Probabilistic latent semantic analysis. In Proceedings of the Fifteenth conference on Uncertainty in artificial intelligence. Morgan Kaufmann Publishers Inc., 289--296. Google ScholarDigital Library
- Yonggang Huang, Jun Zhang, Yongwang Zhao, and Dianfu Ma. 2010. Medical image retrieval with query-dependent feature fusion based on one-class SVM. In Computational Science and Engineering (CSE), 2010 IEEE 13th International Conference on. IEEE, 176--183. Google ScholarDigital Library
- Thomas Lehmann et al. 2003. The IRMA project: A state of the art report on content-based image retrieval in medical applications. In Korea-Germany Workshop on Advanced Medical Image. 161--171.Google Scholar
- Rainer Lienhart, Stefan Romberg, and Eva Hörster. 2009. Multilayer pLSA for multimodal image retrieval. In Proceedings of the ACM International Conference on Image and Video Retrieval. ACM, 9. Google ScholarDigital Library
- David G Lowe. 2004. Distinctive image features from scale-invariant keypoints. International journal of computer vision 60, 2 (2004), 91--110. Google ScholarDigital Library
- Sandy A Napel, Christopher F Beaulieu, Cesar Rodriguez, Jingyu Cui, Jiajing Xu, Ankit Gupta, Daniel Korenblum, Hayit Greenspan, Yongjun Ma, and Daniel L Rubin. 2010. Automated retrieval of CT images of liver lesions on the basis of image similarity: method and preliminary results. Radiology 256, 1 (2010), 243--252.Google ScholarCross Ref
- Trong-Ton Pham, Nicolas Eric Maillot, Joo-Hwee Lim, and J Chevallet. 2007. Latent semantic fusion model for image retrieval and annotation. In Proceedings of the 16th ACM conference on Conference on information and knowledge management. ACM, 439--444. Google ScholarDigital Library
- Joel Pyykkö and Dorota Głowacka. 2016. Interactive content-based image retrieval with deep neural networks. In International Workshop on Symbiotic Interaction. Springer, 77--88.Google Scholar
- Adnan Qayyum, Syed Muhammad Anwar, Muhammad Awais, and Muhammad Majid. 2017. Medical image retrieval using deep convolutional neural network. Neurocomputing 266 (2017), 8--20. Google ScholarDigital Library
- Md Mahmudur Rahman, Sameer K Antani, Rodney L Long, Dina Demner-Fushman, and George R Thoma. 2009. Multi-modal query expansion based on local analysis for medical image retrieval. In MICCAI International Workshop on Medical Content-Based Retrieval for Clinical Decision Support. Springer, 110--119. Google ScholarDigital Library
- Karen Simonyan and Andrew Zisserman. 2014. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 (2014).Google Scholar
- Xiaoying Tai and Weihua Song. 2007. An improved approach based on FCM using feature fusion for medical image retrieval. In Fuzzy Systems and Knowledge Discovery, 2007. FSKD 2007. Fourth International Conference on, Vol. 2. IEEE, 336--342. Google ScholarDigital Library
Index Terms
- An Approach for Multimodal Medical Image Retrieval using Latent Dirichlet Allocation
Recommendations
Multimodal medical image retrieval: image categorization to improve search precision
MIR '10: Proceedings of the international conference on Multimedia information retrievalEffective medical image retrieval can be useful in the clinical care of patients, education and research. Traditionally, image retrieval systems have been text-based, relying on the annotations or captions associated with the images. Although text-based ...
Medical Specialists Retrieval System Using Unified Medical Language System
ICMHI '17: Proceedings of the 1st International Conference on Medical and Health Informatics 2017A large number of doctors and wide range of medical specialties can cause confusion in choosing the right medical specialist. This research aims to build a medical specialists retrieval system that corresponds with the user's disease. To make the system ...
Multimodal medical image retrieval system
In this paper we depict an implemented system for medical image retrieval. Our system performs retrieval based on both textual and visual content, separately and combined, using advanced encoding and quantization techniques. The text-based retrieval ...
Comments