Skip to main content

Advertisement

SpringerLink
Log in

A medical image retrieval scheme with relevance feedback through a medical social network

  • Original Article
  • Published:
Social Network Analysis and Mining Aims and scope Submit manuscript

Abstract

Medical social networking sites enabled multimedia content sharing in large volumes, by allowing physicians and patients to upload their medical images. Moreover, it is necessary to employ new techniques in order to effectively handle and benefit from them. This huge volume of images needs to formulate new types of queries that pose complex questions to medical social network databases. Content-based image retrieval (CBIR) stills an active and efficient research topic to manipulate medical images. In order to palliate this situation, we propose in this paper the integration of a content-based medical image retrieval method through a medical social network, based on an efficient fusion of low-level visual image features (color, shape and texture features), which offers an efficient and flexible precision. A clear application of our CBIR system consists of providing stored images that are visually similar to a new (undiagnosed) one, allowing specialist and patients to check past examination diagnoses from comments and other physicians’ annotations, and to establish, therefore, a new diagnostic or to prepare a new report of an image’s examination. To scale up the performance of the integrated CBIR system, we implement a relevance feedback method. It is an effective method to bridge the semantic gap between low-level visual features and high-level semantic meanings. Experiments show that the proposed medical image retrieval scheme achieves better performance and accuracy in retrieving images. However, we need also to verify whether our approach is considered by the specialists as a potential aid in a real environment. To do so, we evaluate our methodology’s impact in the user’s decision, inquiring the specialists about the degree of confidence in the retrieval system. By analyzing the obtained results, we can argue that the proposed methodology presented a high acceptance regarding the specialists’ interests in the clinical practice domain and can improve the decision-making process during analysis.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

Notes

  1. http://iom.nationalacademies.org/.

  2. http://corp.yougov.com/healthcare/consumers-use-preference-expectations-hospital social-media.

  3. http://www.sobercircle.com.

  4. http://www.sparkpeople.com/.

  5. https://www.fitocracy.com/.

  6. https://www.dacadoo.com/.

  7. http://www.asklepios.com/.

  8. http://www.acc.org/.

  9. http://www.diabspace.com/.

  10. http://www.parlonscancer.ca/.

  11. http://www.renaloo.com/.

  12. http://www.rxspace.com/.

  13. IRMA Homepage (English): http://www.irma-project.org/index_en.php.

  14. http://imagej.nih.gov/ij/.

References

  • Afifi AJ, Ashour WM (2012) Content-based image retrieval using invariant color and texture features. In: International conference on digital image computing techniques and applications (DICTA). IEEE, Fremantle, WA, pp 1–6

  • Agma J, Traina M, André G, Balan R, Bortolotti LM, Traina C Jr (2007) Content-based image retrieval using approximate shape of objects. In: The 17th IEEE symposium on computer-based medical systems (CBMS’07), pp 91–96

  • Aisen AM, Broderick LS, Winer-Muram H, Brodley CE, Kak AC, Pavlopoulou C et al (2003) Automated storage and retrieval of thin section CT images to assist diagnosis: system description and preliminary assessment. Radiology 228(1):265–270

    Article  Google Scholar 

  • Akgül C, Rubin D, Napel S, Beaulieu C, Greenspan H, Acar B (2011) Content-based image retrieval in radiology: current status and future directions. J Digit Imaging 24:208–222

    Article  Google Scholar 

  • Almansoori W, Zarour O, Jarada TN, Karampales P, Rokne J, Alhajj R (2011) Applications of social network construction and analysis in the medical referral process. In: Proceedings of the 2011 IEEE ninth international conference on dependable, autonomic and secure computing (DASC ‘11)

  • Antani S, Long LR, Thoma GR, Lee DJ (2003) Evaluation of shape indexing methods for content-based retrieval of X-ray images. In: Yeung MM, Lienhart RW, Li CS (eds) Proceedings of SPIE 5021, pp 405–416

  • Antani S, Lee D, Long LR, Thoma GR (2004) Evaluation of shape similarity measurement methods for spine X-ray images. J Vis Commun Image Represent 15(3):285–302

    Article  Google Scholar 

  • Arevalillo-Herráez M, Ferri FJ, Moreno-Picot S (2013) A hybrid multi-objective optimization algorithm for content based image retrieval. Appl Soft Comput 13:4358–4369

    Article  Google Scholar 

  • Ashish O, Manpreet S (2012) Content based image retrieval system for medical databases (CBIR-MD)—lucratively tested on endoscopy, dental and skull images. IJCSI Int J Comput Sci Issues 9(1):300–306

    Google Scholar 

  • Bach JR, Fuller C, Gupta A, Hampapur A, Horowitz B, Humphrey R et al (1996) Virage image search engine: an open framework for image management. In: Sethi IK, Jain RC (eds) Proceedings of SPIE, vol 2670, no 1, pp 76–87

  • Bhattacharjee N, Parekh R (2011) Skin texture analysis for medical diagnosis. In: The international conference on communication, computing and security. New York, pp 301–306

  • Bueno R, Ribeiro MX, Traina AJM (2010) Improving medical image retrieval through multi-descriptor similarity functions and association rules. In: IEEE 23rd international symposium on computer-based medical systems (CBMS), pp 309–314

  • Bugatti PH, Ponciano-Silva M, Agma J, Traina M, Traina C Jr, Marques P (2009) Content-based retrieval of medical images: from context to perception. In: 22nd IEEE international symposium on computer-based medical systems (CBMS), pp 1–8

  • Bugatti PH, Ribeiro MX, Traina JM, Traina C Jr (2011) Feature selection guided by perception in medical CBIR systems. In: First IEEE international conference on healthcare informatics, imaging and systems biology, pp 323–330

  • Chechik G, Sharma V, Shalit U, Bengio S (2010) Large scale online learning of image similarity through ranking. J Mach Learn Res 11:1109–1135

    MathSciNet  MATH  Google Scholar 

  • Cox IJ, Miller ML, Minka TP, Papathomas TV, Yianilos PN (2000) The Bayesian image retieval system, PicHunter: theory, implementation and psychophysical experiments. IEEE Tran Image Process 9(1):20–37

    Article  Google Scholar 

  • Daniel RG, Liza SR, Jennifer LK (2013) Dangers and opportunities for social media in medicine. Clin Obstet Gynecol. doi:10.1097/GRF.0b013e318297dc38

    Google Scholar 

  • Datta R, Joshi D, Li J, Wang JZ (2008) Image retrieval: ideas, influences, and trends of the new age. ACM Comput Surv 40(2):5:1–5:60

    Article  Google Scholar 

  • De Oliveira JEE, Machado AMC, Chavez GC, Lopes APB, Deserno TM, De Araujo AA (2010) Mammosys: a content-based image retrieval system using breast density patterns. Comput Methods Programs Biomed 99(3):289–297

    Article  Google Scholar 

  • Doganay S (2014) Healthcare social networks: new choices for doctors, patients. http://www.informationweek.com/healthcare/patient-tools/healthcare-social-networks-new-choices-for-doctors-patients/d/d-id/1234884

  • Doulamis N, Doulamis A (2006) Evaluation of relevance feedback schemes in content-based in retrieval systems. Signal Process Image Commun 21(4):334–357

    Article  MATH  Google Scholar 

  • Fazal M, Baharum B (2013) Analysis of distance metrics in content-based image retrieval using statistical quantized histogram texture features in the DCT domain. J King Saud Univ Comput Inf Sci 25:207–218

    Google Scholar 

  • Feldman DL (2012) Medical social media networks: communicating across the virtual highway. Q J Health Care Pract Risk Manag Infocus 18(1):2–5

    Google Scholar 

  • Flickner M, Sawhney H, Niblack W, Ashley J, Huang Q, Dom B et al (1995) Query by image and video content: the QBIC system. Computer 28(9):23–32

    Article  Google Scholar 

  • Franklin V, Greene S (2007) Sweet talk: a text messaging support system. J Diabetes Nurs 11(1):22–26

    Google Scholar 

  • Goh K-S, Chang EY, Li B (2005) Using one-class and two-class SVMs for multiclass image annotation. IEEE Trans Knowl Data Eng 17(10):1333–1346

    Article  Google Scholar 

  • Gong J, Sun S (2011) Individual doctor recommendation model on medical social network. In: Proceedings of the 7th international conference on advanced data mining and applications (ADMA’11)

  • Greenspan H (2007) Medical image categorization and retrieval. For PACS using the GMM-KL framework. IEEE Trans Inf Technol BioMed 11:190–202

    Article  Google Scholar 

  • Grenier C (2003) The role of intermediate subject to understand the structuring of an organizational network of actors and technology—case of a care network. In: Proceedings of the 9th conference of the association information and management, Grenoble

  • Güld MO, Thies C, Fischer B, Lehmann TM (2007) A generic concept for the implementation of medical image retrieval systems. Int Med Inform 76(2–3):252–259

    Article  Google Scholar 

  • Harishchandra H, Mushigeri S, Niranjan UC (2014) Medical image retrieval–performance comparison using texture features. Int J Eng Res Dev 9(9):30–34

    Google Scholar 

  • Hoi SCHH, Jin R, Zhu JK, Lyu MR (2009) Semi-supervised SVM batch mode active learning and its applications to image retrieval. ACM Trans Inf Syst 27(3):1–29

    Article  Google Scholar 

  • Hsu W, Antani S, Long LR, Neve L, Thoma GR (2009) SPIRS: a web based image retrieval system for large biomedical databases. Int J Med Inform 78(1):13–24

    Article  Google Scholar 

  • Hu W, Xie N, Li L, Zeng X (2011) A survey on visual content-based video indexing and retrieval. IEEE Trans Syst Man Cybern C Appl Rev 41(6):797–819

    Article  Google Scholar 

  • Iakovidis D, Pelekis N, Kotsifakos E, Kopanakis I, Karanikas H, Theodoridis Y (2009) A pattern similarity scheme for medical imag retrieval. IEEE Trans Inf Technol Biomed 13(4):442–509

    Article  Google Scholar 

  • ISO, IEC 25010 (2011) Systems and software engineering-Systems and software Quality Requirements and Evaluation (SQuaRE)-System and software quality models. International Standards Organization, Geneva

    Google Scholar 

  • John C, Kazunori O (2012) A comparative study of similarity measures for content-based medical image retrieval. http://ceur-ws.org/Vol-1178/CLEF2012wn-ImageCLEF-CollinsEt2012.pdf2012

  • Keysers D, Dahmen J, Ney H, Wein BB, Lehmann TM (2003) Statistica framework for model-based image retrieval in medical applications. J Electron Imaging 12(1):59–68

    Article  Google Scholar 

  • Komali A et al (2012) 3D color feature extraction in content-based image retrieval. Int J Soft Comput Eng (IJSCE) 2(3):560–563

    Google Scholar 

  • Kumar A, Kim J, Cai W, Fulham M, Feng D (2013) Content-based medical image retrieval: a survey of applications to multidimensional and multimodality data. J Digit Imaging. doi:10.1007/s10278-013-9619-2

    Google Scholar 

  • Lee DJ, Antani S, Long LR (2003) Similarity measurement using polygon curve representation and Fourier descriptors for shape-based vertebral image retrieval. In: Sonka M, Fitzpatrick JM (eds) Proceedings of SPIE, vol 5032, pp 1283–1291

  • Lee DJ, Antani S, Chang Y, Gledhill K, Long LR, Christensen P (2009) CBIR of spine X-ray images on intervertebral disc space and shape profiles using feature ranking and voting consensus. Data Knowl Eng 68(12):1359–1369

    Article  Google Scholar 

  • Lew MS, Sebe N, Djeraba C, Jain R (2006) Content-based multimedia information retrieval: state of the art and challenges. ACM Trans Multimed Comput Commun Appl 2(1):1–19

    Article  Google Scholar 

  • Li J (2014) Data protection in healthcare social networks. J IEEE Softw 31(1):46–53

    Article  Google Scholar 

  • Liu Y, Zhang D, Lu G, Ma W-Y (2007) A survey of content based image retrieval with high-level semantics. Pattern Recognit Lett 40(1):262–282

    Article  MATH  Google Scholar 

  • MacArthur SD, Brodley CE, Shyu CR (2000) Relevance feedback decision trees in content-based image retrieval. In: Proceedings of the IEEE work-shop content-based access of image and video libraries, pp 68–72

  • Manning CD, Raghavan P, Schutze H (2008) Introduction to information retrieval. Cambridge University Press, New York

    Book  MATH  Google Scholar 

  • Messaoudi A, Bouslimi R, Akaichi J (2013) Indexing medical images based on collaborative experts reports. Int J Comput Appl 70(5):1–9

    Google Scholar 

  • Mezaris V, Kompatsiaris I, Strintzis MG (2005) An ontology approach to object based image retrieval. In: Proceedings of the international conference on image processing, pp 511–514

  • Müller H, Rosset A, Garcia A, Vallée JP, Geissbuhler A (2005) Benefits of content-based visual data access in radiology. Radiographics 25(3):849–858

    Article  Google Scholar 

  • Nandagopalan S, Adiga BS, Deepak N (2008) A universal model for content-based image retrieval. World Acad Sci Eng Technol 46:644–647

    Google Scholar 

  • Napel SA, Beaulieu CF, Rodriguez C, Cui J, Xu J, Gupta A et al (2010) Automated retrieval of CT images of liver lesions on the basis of image similarity: method and preliminary results. Radiology 256(1):243–252

    Article  Google Scholar 

  • Patil PB, Kokare MB (2011) Relevance feedback in content based image retrieval: a review. J Appli Comput Sci Math 10:41–47

    Google Scholar 

  • Pentland A, Picard RW, Sclaroff S (1996) Photobook: content-based manipulation of image databases. Int J Comput Vis 18:233–254

    Article  Google Scholar 

  • Qian X, Tagare HD, Fulbright RK, Long R, Antani S (2010) Optimal embedding for shape indexing in medical image databases. Med Image Anal 14(3):243–254

    Article  Google Scholar 

  • Rajakumar K, Muttan S (2013) MRI image retrieval using Wavelet with Mahalanobis distance measurement. J Electr Eng Technol 8(5):1188–1193

    Article  Google Scholar 

  • Ramamurthy B, Chandran KR (2012) Content based medical image retrieval with texture content using gray level co-occurrence matrix and k-means clustering algorithms. J Comput Sci 8(7):1070–1076

    Article  Google Scholar 

  • Ramamurthy B, Chandran KR, Meenakshi VR, Shilpa V (2012) CBMIR: content based medical image retrieval system using texture and intensity for dental images. In: International conference eco-friendly computing and communication systems, ICECCS, vol 305, pp 125–134

  • Rocchio JJ (1971) Relevance feedback in information retrieval: SMART retrieval system, 1st edn. Prentice Hall, Upper Saddle River, pp 323–341

    Google Scholar 

  • Rui Y, Huang TS, Chang SF (1999) Image retrieval: current techniques, promising directions, and open issues. J Vis Commun Image Represent 10(1):39–62

    Article  Google Scholar 

  • Selvarani G, Annadurai S (2007) Medical image retrieval by combining low level features and DICOM features. In: International IEEE conference on computational intelligence and multimedia applications, pp 587–589

  • Seng WC, Mirisaee SH (2009) A content-based retrieval system for blood cells images. In: International IEEE conference on future computer and communication, pp 412–415

  • Shanmugapriya N, Nallusamy R (2014) A new content based image retrieval system using GMM and relevance feedback. J Comput Sci 10(2):330–340

    Article  Google Scholar 

  • Shyu CR, Brodley CE, Kak AC, Kosaka A, Aisen AM, Broderick LS (1999) ASSERT: a physician-in-the-loop content-based retrieval system for HRCT image databases. Comput Vision Image Underst 75(1–2):111–132

    Article  Google Scholar 

  • Sidong L, Lei J, Weidong C, Lingfeng W, Eberl S, Fulham MJ, Dagan F (2010) Localized multiscale texture based retrieval of neurological image. In: IEEE 23rd international symposium on computer-based medical systems (CBMS), pp 243–248

  • Singh P, Singh S, Kaur G (2009) Efficient techniques used in CBMIR for medical image retrievals. Proc World Acad Sci Eng Technol 38:434–437

    Google Scholar 

  • Singh J, Kaleka JS, Sharma R (2012) Different approaches of CBIR techniques. Int J Comput Distributed Syst 1(2):76–78

    Google Scholar 

  • Smeulders A, Worring M, Santini S, Gupta A, Jain R (2000) Content based image retrieval at the end of the early years. IEEE TransPattern Anal Mach Intell 22(12):1349–1380

    Article  Google Scholar 

  • Song Y (2012) Image Analysis for Automatic Phenotyping Measurements. Biomathematics and Statistics Scotland (BioSS). Wageningen

  • Stojmenovic M, Nayak A (2008) Measuring the related properties of linearity and elongation of point sets. In: 13th Iberoamerican congress on pattern recognition, CIARP 2008. Havana, Cuba, pp 102–111

  • Stojmenovic M, Jevremovic A, Nayak A (2013) Fast iris detection via shape based circularity. In: 8th IEEE conference on industrial electronics and applications (ICIEA), pp 747–752

  • Su Z, Zhang H, Li S, Ma S (2003) Relevance feedback in content-based image retrieval: Bayesian framework, feature subspaces, and progressive learning. IEEE Trans Image Process 12(8):924–936

    Article  Google Scholar 

  • Swarnambiga A, Vasuki SM (2013) Distance measures for medical image retrieval. Int J Imaging Syst Technol 23:9–21

    Article  Google Scholar 

  • Tieu K, Viola P (2003) Boosting image retrieval. In: Proceedings of the IEEE conference on computer vision pattern recognition, pp 228–235

  • Wang M, Hua XS (2011) Active learning in multimedia annotation and retrieval: a survey. ACM Trans Intell Syst Technol 2(2):10–31

    Article  MathSciNet  Google Scholar 

  • Wang M, Li H, Tao D, Lu K, Wu X (2012) Multimodal graph-based reranking for web image search. IEEE Trans Image Process 21(11):4649–4661

    Article  MathSciNet  Google Scholar 

  • Wang X-Y, Li Y-W, Yang H-Y, Chen J-W (2014) An image retrieval scheme with relevance feedback using feature reconstruction and SVM reclassification. Neurocomputing 127:214–230

    Article  Google Scholar 

  • Wu C, Tai X (2009) Application of gray level variation statistic in gastroscopic image retrieval. In: Eighth IEEE/ACIS international conference on computer and information science, pp 342–346

  • Xiang-Yang W, Bei-Bei Z, Hong-Ying Y (2012) Content-based image retrieval by integrating color and texture features. Multimed Tools Appl 68(3):545–569

    Google Scholar 

  • Xie Y, Chen Z, Cheng Y, Zhang K, Agrawal A, Liao WK, Choudhary A (2013) Detecting and tracking disease outbreaks by mining social media data. In: Proceedings of the twenty-third international joint conference on artificial intelligence (IJCAI’13)

  • Xin J, Jin JS (2004) Relevance feedback for content-based image retrieval using Bayesian network. In: Proceedings of the pan-sydney area workshop on visual information processing (VIP ‘05). Australian Computer Society, Inc Darlinghurst Australia, pp 91–94

  • Xu X, Lee DJ, Antani S, Long L (2008) A spine X-ray image retrieval system using partial shape matching. IEEE Trans Inf Technol Biomed 12(1):100–108

    Article  Google Scholar 

  • Yogapriya J, Ila V (2013) An integrated framework based on texture features, cuckoo search and relevance vector machine for medical image retrieval system. Am J Appl Sci 10(11):1398–1412

    Article  Google Scholar 

  • Yue J, Li Z, Liu L, Fu Z (2010) Content-based image retrieval using color and texture fused features. Math Comput Model 54:1121–1127

    Article  Google Scholar 

  • Zeyad SY, Dzulkifli M, Tanzila S, Mohammed HA, Amjad R, Al-R Mznah, Al-D Abdullah (2014) Content-based image retrieval using PSO and k-means clustering algorithm. Arab J Geosci 8(8):6211–6224. doi:10.1007/s12517-014-1584-7

    Google Scholar 

  • Zhang D, Lu G (2004) Review of shape representation and description techniques. Pattern Recognit 37(1):1–19

    Article  Google Scholar 

  • Zhang J, Ye L (2010) Series feature aggregation for content-based image retrieval. Comput Electr Eng 36(4):691–701

    Article  MATH  Google Scholar 

  • Zhang G, Ma ZM, He Y, Zhao T (2008) Texture characteristic extraction for dominant directions in content-based medical image retrieval. In: IEEE international conference on biomedical engineering and informatics (BMEI), pp 253–257

  • Zhang D, Islam MdM, Lu G (2012) A review on automatic image annotation techniques. Pattern Recognit 45:346–362

    Article  Google Scholar 

  • Zhi W, Wenwu Z, Peng C, Lifeng S, Shiqiang Y (2013) Social media recommendation. Soc Media Retr Comput Commun Netw. doi:10.1007/978-1-4471-4555-43

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Computer Sciences, ISG, BESTMOD, Tunis, Tunisia

    Mouhamed Gaith Ayadi, Riadh Bouslimi & Jalel Akaichi

Authors
  1. Mouhamed Gaith Ayadi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Riadh Bouslimi
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jalel Akaichi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Mouhamed Gaith Ayadi.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Ayadi, M.G., Bouslimi, R. & Akaichi, J. A medical image retrieval scheme with relevance feedback through a medical social network. Soc. Netw. Anal. Min. 6, 53 (2016). https://doi.org/10.1007/s13278-016-0362-9

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s13278-016-0362-9

Keywords

Search

Navigation