Abstract
Medical social networking sites enabled multimedia content sharing in large volumes, by allowing physicians and patients to upload their medical images. Moreover, it is necessary to employ new techniques in order to effectively handle and benefit from them. This huge volume of images needs to formulate new types of queries that pose complex questions to medical social network databases. Content-based image retrieval (CBIR) stills an active and efficient research topic to manipulate medical images. In order to palliate this situation, we propose in this paper the integration of a content-based medical image retrieval method through a medical social network, based on an efficient fusion of low-level visual image features (color, shape and texture features), which offers an efficient and flexible precision. A clear application of our CBIR system consists of providing stored images that are visually similar to a new (undiagnosed) one, allowing specialist and patients to check past examination diagnoses from comments and other physicians’ annotations, and to establish, therefore, a new diagnostic or to prepare a new report of an image’s examination. To scale up the performance of the integrated CBIR system, we implement a relevance feedback method. It is an effective method to bridge the semantic gap between low-level visual features and high-level semantic meanings. Experiments show that the proposed medical image retrieval scheme achieves better performance and accuracy in retrieving images. However, we need also to verify whether our approach is considered by the specialists as a potential aid in a real environment. To do so, we evaluate our methodology’s impact in the user’s decision, inquiring the specialists about the degree of confidence in the retrieval system. By analyzing the obtained results, we can argue that the proposed methodology presented a high acceptance regarding the specialists’ interests in the clinical practice domain and can improve the decision-making process during analysis.











Similar content being viewed by others
Notes
IRMA Homepage (English): http://www.irma-project.org/index_en.php.
References
Afifi AJ, Ashour WM (2012) Content-based image retrieval using invariant color and texture features. In: International conference on digital image computing techniques and applications (DICTA). IEEE, Fremantle, WA, pp 1–6
Agma J, Traina M, André G, Balan R, Bortolotti LM, Traina C Jr (2007) Content-based image retrieval using approximate shape of objects. In: The 17th IEEE symposium on computer-based medical systems (CBMS’07), pp 91–96
Aisen AM, Broderick LS, Winer-Muram H, Brodley CE, Kak AC, Pavlopoulou C et al (2003) Automated storage and retrieval of thin section CT images to assist diagnosis: system description and preliminary assessment. Radiology 228(1):265–270
Akgül C, Rubin D, Napel S, Beaulieu C, Greenspan H, Acar B (2011) Content-based image retrieval in radiology: current status and future directions. J Digit Imaging 24:208–222
Almansoori W, Zarour O, Jarada TN, Karampales P, Rokne J, Alhajj R (2011) Applications of social network construction and analysis in the medical referral process. In: Proceedings of the 2011 IEEE ninth international conference on dependable, autonomic and secure computing (DASC ‘11)
Antani S, Long LR, Thoma GR, Lee DJ (2003) Evaluation of shape indexing methods for content-based retrieval of X-ray images. In: Yeung MM, Lienhart RW, Li CS (eds) Proceedings of SPIE 5021, pp 405–416
Antani S, Lee D, Long LR, Thoma GR (2004) Evaluation of shape similarity measurement methods for spine X-ray images. J Vis Commun Image Represent 15(3):285–302
Arevalillo-Herráez M, Ferri FJ, Moreno-Picot S (2013) A hybrid multi-objective optimization algorithm for content based image retrieval. Appl Soft Comput 13:4358–4369
Ashish O, Manpreet S (2012) Content based image retrieval system for medical databases (CBIR-MD)—lucratively tested on endoscopy, dental and skull images. IJCSI Int J Comput Sci Issues 9(1):300–306
Bach JR, Fuller C, Gupta A, Hampapur A, Horowitz B, Humphrey R et al (1996) Virage image search engine: an open framework for image management. In: Sethi IK, Jain RC (eds) Proceedings of SPIE, vol 2670, no 1, pp 76–87
Bhattacharjee N, Parekh R (2011) Skin texture analysis for medical diagnosis. In: The international conference on communication, computing and security. New York, pp 301–306
Bueno R, Ribeiro MX, Traina AJM (2010) Improving medical image retrieval through multi-descriptor similarity functions and association rules. In: IEEE 23rd international symposium on computer-based medical systems (CBMS), pp 309–314
Bugatti PH, Ponciano-Silva M, Agma J, Traina M, Traina C Jr, Marques P (2009) Content-based retrieval of medical images: from context to perception. In: 22nd IEEE international symposium on computer-based medical systems (CBMS), pp 1–8
Bugatti PH, Ribeiro MX, Traina JM, Traina C Jr (2011) Feature selection guided by perception in medical CBIR systems. In: First IEEE international conference on healthcare informatics, imaging and systems biology, pp 323–330
Chechik G, Sharma V, Shalit U, Bengio S (2010) Large scale online learning of image similarity through ranking. J Mach Learn Res 11:1109–1135
Cox IJ, Miller ML, Minka TP, Papathomas TV, Yianilos PN (2000) The Bayesian image retieval system, PicHunter: theory, implementation and psychophysical experiments. IEEE Tran Image Process 9(1):20–37
Daniel RG, Liza SR, Jennifer LK (2013) Dangers and opportunities for social media in medicine. Clin Obstet Gynecol. doi:10.1097/GRF.0b013e318297dc38
Datta R, Joshi D, Li J, Wang JZ (2008) Image retrieval: ideas, influences, and trends of the new age. ACM Comput Surv 40(2):5:1–5:60
De Oliveira JEE, Machado AMC, Chavez GC, Lopes APB, Deserno TM, De Araujo AA (2010) Mammosys: a content-based image retrieval system using breast density patterns. Comput Methods Programs Biomed 99(3):289–297
Doganay S (2014) Healthcare social networks: new choices for doctors, patients. http://www.informationweek.com/healthcare/patient-tools/healthcare-social-networks-new-choices-for-doctors-patients/d/d-id/1234884
Doulamis N, Doulamis A (2006) Evaluation of relevance feedback schemes in content-based in retrieval systems. Signal Process Image Commun 21(4):334–357
Fazal M, Baharum B (2013) Analysis of distance metrics in content-based image retrieval using statistical quantized histogram texture features in the DCT domain. J King Saud Univ Comput Inf Sci 25:207–218
Feldman DL (2012) Medical social media networks: communicating across the virtual highway. Q J Health Care Pract Risk Manag Infocus 18(1):2–5
Flickner M, Sawhney H, Niblack W, Ashley J, Huang Q, Dom B et al (1995) Query by image and video content: the QBIC system. Computer 28(9):23–32
Franklin V, Greene S (2007) Sweet talk: a text messaging support system. J Diabetes Nurs 11(1):22–26
Goh K-S, Chang EY, Li B (2005) Using one-class and two-class SVMs for multiclass image annotation. IEEE Trans Knowl Data Eng 17(10):1333–1346
Gong J, Sun S (2011) Individual doctor recommendation model on medical social network. In: Proceedings of the 7th international conference on advanced data mining and applications (ADMA’11)
Greenspan H (2007) Medical image categorization and retrieval. For PACS using the GMM-KL framework. IEEE Trans Inf Technol BioMed 11:190–202
Grenier C (2003) The role of intermediate subject to understand the structuring of an organizational network of actors and technology—case of a care network. In: Proceedings of the 9th conference of the association information and management, Grenoble
Güld MO, Thies C, Fischer B, Lehmann TM (2007) A generic concept for the implementation of medical image retrieval systems. Int Med Inform 76(2–3):252–259
Harishchandra H, Mushigeri S, Niranjan UC (2014) Medical image retrieval–performance comparison using texture features. Int J Eng Res Dev 9(9):30–34
Hoi SCHH, Jin R, Zhu JK, Lyu MR (2009) Semi-supervised SVM batch mode active learning and its applications to image retrieval. ACM Trans Inf Syst 27(3):1–29
Hsu W, Antani S, Long LR, Neve L, Thoma GR (2009) SPIRS: a web based image retrieval system for large biomedical databases. Int J Med Inform 78(1):13–24
Hu W, Xie N, Li L, Zeng X (2011) A survey on visual content-based video indexing and retrieval. IEEE Trans Syst Man Cybern C Appl Rev 41(6):797–819
Iakovidis D, Pelekis N, Kotsifakos E, Kopanakis I, Karanikas H, Theodoridis Y (2009) A pattern similarity scheme for medical imag retrieval. IEEE Trans Inf Technol Biomed 13(4):442–509
ISO, IEC 25010 (2011) Systems and software engineering-Systems and software Quality Requirements and Evaluation (SQuaRE)-System and software quality models. International Standards Organization, Geneva
John C, Kazunori O (2012) A comparative study of similarity measures for content-based medical image retrieval. http://ceur-ws.org/Vol-1178/CLEF2012wn-ImageCLEF-CollinsEt2012.pdf2012
Keysers D, Dahmen J, Ney H, Wein BB, Lehmann TM (2003) Statistica framework for model-based image retrieval in medical applications. J Electron Imaging 12(1):59–68
Komali A et al (2012) 3D color feature extraction in content-based image retrieval. Int J Soft Comput Eng (IJSCE) 2(3):560–563
Kumar A, Kim J, Cai W, Fulham M, Feng D (2013) Content-based medical image retrieval: a survey of applications to multidimensional and multimodality data. J Digit Imaging. doi:10.1007/s10278-013-9619-2
Lee DJ, Antani S, Long LR (2003) Similarity measurement using polygon curve representation and Fourier descriptors for shape-based vertebral image retrieval. In: Sonka M, Fitzpatrick JM (eds) Proceedings of SPIE, vol 5032, pp 1283–1291
Lee DJ, Antani S, Chang Y, Gledhill K, Long LR, Christensen P (2009) CBIR of spine X-ray images on intervertebral disc space and shape profiles using feature ranking and voting consensus. Data Knowl Eng 68(12):1359–1369
Lew MS, Sebe N, Djeraba C, Jain R (2006) Content-based multimedia information retrieval: state of the art and challenges. ACM Trans Multimed Comput Commun Appl 2(1):1–19
Li J (2014) Data protection in healthcare social networks. J IEEE Softw 31(1):46–53
Liu Y, Zhang D, Lu G, Ma W-Y (2007) A survey of content based image retrieval with high-level semantics. Pattern Recognit Lett 40(1):262–282
MacArthur SD, Brodley CE, Shyu CR (2000) Relevance feedback decision trees in content-based image retrieval. In: Proceedings of the IEEE work-shop content-based access of image and video libraries, pp 68–72
Manning CD, Raghavan P, Schutze H (2008) Introduction to information retrieval. Cambridge University Press, New York
Messaoudi A, Bouslimi R, Akaichi J (2013) Indexing medical images based on collaborative experts reports. Int J Comput Appl 70(5):1–9
Mezaris V, Kompatsiaris I, Strintzis MG (2005) An ontology approach to object based image retrieval. In: Proceedings of the international conference on image processing, pp 511–514
Müller H, Rosset A, Garcia A, Vallée JP, Geissbuhler A (2005) Benefits of content-based visual data access in radiology. Radiographics 25(3):849–858
Nandagopalan S, Adiga BS, Deepak N (2008) A universal model for content-based image retrieval. World Acad Sci Eng Technol 46:644–647
Napel SA, Beaulieu CF, Rodriguez C, Cui J, Xu J, Gupta A et al (2010) Automated retrieval of CT images of liver lesions on the basis of image similarity: method and preliminary results. Radiology 256(1):243–252
Patil PB, Kokare MB (2011) Relevance feedback in content based image retrieval: a review. J Appli Comput Sci Math 10:41–47
Pentland A, Picard RW, Sclaroff S (1996) Photobook: content-based manipulation of image databases. Int J Comput Vis 18:233–254
Qian X, Tagare HD, Fulbright RK, Long R, Antani S (2010) Optimal embedding for shape indexing in medical image databases. Med Image Anal 14(3):243–254
Rajakumar K, Muttan S (2013) MRI image retrieval using Wavelet with Mahalanobis distance measurement. J Electr Eng Technol 8(5):1188–1193
Ramamurthy B, Chandran KR (2012) Content based medical image retrieval with texture content using gray level co-occurrence matrix and k-means clustering algorithms. J Comput Sci 8(7):1070–1076
Ramamurthy B, Chandran KR, Meenakshi VR, Shilpa V (2012) CBMIR: content based medical image retrieval system using texture and intensity for dental images. In: International conference eco-friendly computing and communication systems, ICECCS, vol 305, pp 125–134
Rocchio JJ (1971) Relevance feedback in information retrieval: SMART retrieval system, 1st edn. Prentice Hall, Upper Saddle River, pp 323–341
Rui Y, Huang TS, Chang SF (1999) Image retrieval: current techniques, promising directions, and open issues. J Vis Commun Image Represent 10(1):39–62
Selvarani G, Annadurai S (2007) Medical image retrieval by combining low level features and DICOM features. In: International IEEE conference on computational intelligence and multimedia applications, pp 587–589
Seng WC, Mirisaee SH (2009) A content-based retrieval system for blood cells images. In: International IEEE conference on future computer and communication, pp 412–415
Shanmugapriya N, Nallusamy R (2014) A new content based image retrieval system using GMM and relevance feedback. J Comput Sci 10(2):330–340
Shyu CR, Brodley CE, Kak AC, Kosaka A, Aisen AM, Broderick LS (1999) ASSERT: a physician-in-the-loop content-based retrieval system for HRCT image databases. Comput Vision Image Underst 75(1–2):111–132
Sidong L, Lei J, Weidong C, Lingfeng W, Eberl S, Fulham MJ, Dagan F (2010) Localized multiscale texture based retrieval of neurological image. In: IEEE 23rd international symposium on computer-based medical systems (CBMS), pp 243–248
Singh P, Singh S, Kaur G (2009) Efficient techniques used in CBMIR for medical image retrievals. Proc World Acad Sci Eng Technol 38:434–437
Singh J, Kaleka JS, Sharma R (2012) Different approaches of CBIR techniques. Int J Comput Distributed Syst 1(2):76–78
Smeulders A, Worring M, Santini S, Gupta A, Jain R (2000) Content based image retrieval at the end of the early years. IEEE TransPattern Anal Mach Intell 22(12):1349–1380
Song Y (2012) Image Analysis for Automatic Phenotyping Measurements. Biomathematics and Statistics Scotland (BioSS). Wageningen
Stojmenovic M, Nayak A (2008) Measuring the related properties of linearity and elongation of point sets. In: 13th Iberoamerican congress on pattern recognition, CIARP 2008. Havana, Cuba, pp 102–111
Stojmenovic M, Jevremovic A, Nayak A (2013) Fast iris detection via shape based circularity. In: 8th IEEE conference on industrial electronics and applications (ICIEA), pp 747–752
Su Z, Zhang H, Li S, Ma S (2003) Relevance feedback in content-based image retrieval: Bayesian framework, feature subspaces, and progressive learning. IEEE Trans Image Process 12(8):924–936
Swarnambiga A, Vasuki SM (2013) Distance measures for medical image retrieval. Int J Imaging Syst Technol 23:9–21
Tieu K, Viola P (2003) Boosting image retrieval. In: Proceedings of the IEEE conference on computer vision pattern recognition, pp 228–235
Wang M, Hua XS (2011) Active learning in multimedia annotation and retrieval: a survey. ACM Trans Intell Syst Technol 2(2):10–31
Wang M, Li H, Tao D, Lu K, Wu X (2012) Multimodal graph-based reranking for web image search. IEEE Trans Image Process 21(11):4649–4661
Wang X-Y, Li Y-W, Yang H-Y, Chen J-W (2014) An image retrieval scheme with relevance feedback using feature reconstruction and SVM reclassification. Neurocomputing 127:214–230
Wu C, Tai X (2009) Application of gray level variation statistic in gastroscopic image retrieval. In: Eighth IEEE/ACIS international conference on computer and information science, pp 342–346
Xiang-Yang W, Bei-Bei Z, Hong-Ying Y (2012) Content-based image retrieval by integrating color and texture features. Multimed Tools Appl 68(3):545–569
Xie Y, Chen Z, Cheng Y, Zhang K, Agrawal A, Liao WK, Choudhary A (2013) Detecting and tracking disease outbreaks by mining social media data. In: Proceedings of the twenty-third international joint conference on artificial intelligence (IJCAI’13)
Xin J, Jin JS (2004) Relevance feedback for content-based image retrieval using Bayesian network. In: Proceedings of the pan-sydney area workshop on visual information processing (VIP ‘05). Australian Computer Society, Inc Darlinghurst Australia, pp 91–94
Xu X, Lee DJ, Antani S, Long L (2008) A spine X-ray image retrieval system using partial shape matching. IEEE Trans Inf Technol Biomed 12(1):100–108
Yogapriya J, Ila V (2013) An integrated framework based on texture features, cuckoo search and relevance vector machine for medical image retrieval system. Am J Appl Sci 10(11):1398–1412
Yue J, Li Z, Liu L, Fu Z (2010) Content-based image retrieval using color and texture fused features. Math Comput Model 54:1121–1127
Zeyad SY, Dzulkifli M, Tanzila S, Mohammed HA, Amjad R, Al-R Mznah, Al-D Abdullah (2014) Content-based image retrieval using PSO and k-means clustering algorithm. Arab J Geosci 8(8):6211–6224. doi:10.1007/s12517-014-1584-7
Zhang D, Lu G (2004) Review of shape representation and description techniques. Pattern Recognit 37(1):1–19
Zhang J, Ye L (2010) Series feature aggregation for content-based image retrieval. Comput Electr Eng 36(4):691–701
Zhang G, Ma ZM, He Y, Zhao T (2008) Texture characteristic extraction for dominant directions in content-based medical image retrieval. In: IEEE international conference on biomedical engineering and informatics (BMEI), pp 253–257
Zhang D, Islam MdM, Lu G (2012) A review on automatic image annotation techniques. Pattern Recognit 45:346–362
Zhi W, Wenwu Z, Peng C, Lifeng S, Shiqiang Y (2013) Social media recommendation. Soc Media Retr Comput Commun Netw. doi:10.1007/978-1-4471-4555-43
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Ayadi, M.G., Bouslimi, R. & Akaichi, J. A medical image retrieval scheme with relevance feedback through a medical social network. Soc. Netw. Anal. Min. 6, 53 (2016). https://doi.org/10.1007/s13278-016-0362-9
Received:
Revised:
Accepted:
Published:
DOI: https://doi.org/10.1007/s13278-016-0362-9