Skip to main content
SpringerLink
Log in

Medical image retrieval based on unclean image bags

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Traditional content-based image retrieval (CBIR) scheme with assumption of independent individual images in large-scale collections suffers from poor retrieval performance. In medical applications, images usually exist in the form of image bags and each bag includes multiple relevant images of the same perceptual meaning. In this paper, based on these natural image bags, we explore a new scheme to improve the performance of medical image retrieval. It is feasible and efficient to search the bag-based medical image collection by providing a query bag. However, there is a critical problem of noisy images which may present in image bags and severely affect the retrieval performance. A new three-stage solution is proposed to perform the retrieval and handle the noisy images. In stage 1, in order to alleviate the influence of noisy images, we associate each image in the image bags with a relevance degree. In stage 2, a novel similarity aggregation method is proposed to incorporate image relevance and feature importance into the similarity computation process. In stage 3, we obtain the final image relevance in an adaptive way which can consider both image bag similarity and individual image similarity. The experiments demonstrate that the proposed approach can improve the image retrieval performance significantly.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

Notes

  1. http://mirc.rsna.org

  2. http://www.irma-project.org

  3. http://rad.usuhs.edu/medpix

References

  1. Arevalillo-Herráez M, Ferri FJ, Domingo J (2010) A naive relevance feedback model for content-based image retrieval using multiple similarity measures. Pattern Recogn 43(3):619–629

    Article  MATH  Google Scholar 

  2. Chang CC, Lin CJ (2012) LIBSVM: a library for support vector machines. National Taiwan University. http://www.csie.ntu.edu.tw/~cjlin/libsvm/. Accessed 15 Sept 2012

  3. Chang SF, Sikora T, Purl A (2001) Overview of the MPEG-7 standard. IEEE Trans Circuits Syst Video Technol 11(6):688–695

    Article  Google Scholar 

  4. Donald K, Smeaton A (2005) A comparison of score, rank and probability-based fusion methods for video shot retrieval. In: The 4th international conference on image and video retrieval. Singapore, pp 61–70

  5. Greenspan H, Pinhas AT (2007) Medical image categorization and retrieval for pacs using the GMM-KL framework. IEEE Trans Inf Technol Biomed 11(2):190–202

    Article  Google Scholar 

  6. Huang YH, Zhang J, Zhao YW, Ma DF (2010) Medical image retrieval with query-dependent feature fusion based on one-class SVM. In: The 13th International Conference on Computational Science and Engineering. Hongkong, China, pp 176–183

  7. Iakovidis DK, Pelekis N, Kotsifakos EE, Kopanakis I, Karanikas H, Theodoridis Y (2009) A pattern similarity scheme for medical image retrieval. IEEE Trans Inf Technol Biomed 13(4):442–450

    Article  Google Scholar 

  8. Kong J, Gollub RL, Polich G, Kirsch I, LaViolette P, Vangel M, Rosen B, Kaptchuk TJ (2008) A functional magnetic resonance imaging study on the neural mechanisms of hyperalgesic nocebo effect. J Neurosci 28(49):13354–13362

    Article  Google Scholar 

  9. Kushki A, Androutsos P, Plataniotis KN, Venetsanopoulos AN (2004) Retrieval of images from artistic repositories using a decision fusion framework. IEEE Trans Image Process 13(3):277

    Article  Google Scholar 

  10. Lehmann TM, Schubert H, Ott B, Leisten M (2012) Image retrieval in medical applications. RWTH Aachen University. http://ganymed.imib.rwth-aachen.de/irma/datasets. Accessed 15 Sept 2012

  11. Lew MS, Sebe N, Djeraba C, Jain R (2006) Content-based multimedia information retrieval: State of the art and challenges. ACM Trans Multimed Comput Commun Appl 2(1):1–19

    Google Scholar 

  12. Li J, Allinson N, Tao DC, Li XL (2006) Multitraining support vector machine for image retrieval. IEEE Trans Image Process 15(11):3597–3601

    Article  Google Scholar 

  13. Lin CF, Wang SD (2002) Fuzzy support vector machines. IEEE Trans Neural Netw 13(2):464–471

    Google Scholar 

  14. Machado CJ, Snyder AZ, Cherry SR, Lavenex P, Amaral DG (2008) Effects of neonatal amygdala or hippocampus lesions on resting brain metabolism in the macaque monkey: a microPET imaging study. Neuroimage 39(2):832–846

    Article  Google Scholar 

  15. Mildenberger P, Eichelberg M, Martin E (2002) Introduction to the DICOM standard. Eur Radiol 12(4):920–927

    Article  Google Scholar 

  16. Muller H, Michoux N, Bandon D, Geissbuhler A (2004) A review of content-based image retrieval systems in medical applications–clinical benefits and future directions. Int J Med Inform 73(1):1–23

    Article  Google Scholar 

  17. Müller H, Müller W, Squire DMG, Marchand-Maillet S, Pun T (2001) Performance evaluation in content-based image retrieval: Overview and proposals. Pattern Recogn Lett 22(5):593–601

    Article  MATH  Google Scholar 

  18. Qin T, Zhang XD, Liu TY, Wang DS, Ma WY, Zhang HJ (2008) An active feedback framework for image retrieval. Pattern Recogn Lett 29(5):637–646

    Article  Google Scholar 

  19. Rahman M, Antani SK, Thoma GR (2011) A biomedical image retrieval framework based on classification-driven image filtering and similarity fusion. In: IEEE international symposium on biomedical imaging: from nano to macro. Chicago, IL, pp 1905–1908

  20. Rahman MM, Antani SK, Thoma GR (2011) A learning-based similarity fusion and filtering approach for biomedical image retrieval using SVM classification and relevance feedback. IEEE Trans Inf Technol Biomed 15(4):640–646

    Article  Google Scholar 

  21. Rahman MM, Bhattacharya P, Desai BC (2007) A framework for medical image retrieval using machine learning and statistical similarity matching techniques with relevance feedback. IEEE Trans Inf Technol Biomed 11(1):58–69

    Article  Google Scholar 

  22. Rahman MM, Desai BC, Bhattacharya P (2008) Medical image retrieval with probabilistic multi-class support vector machine classifiers and adaptive similarity fusion. Comput Med Imaging Graph 32(2):95–108

    Article  Google Scholar 

  23. Ru L, Ma S, Lu J (2005) Feature fusion based on the average precision in image retrieval. Journal of Computer Research and Development 42(9):1640–1646

    Article  Google Scholar 

  24. Scott G, Shyu CR (2007) Knowledge-driven multidimensional indexing structure for biomedical media database retrieval. IEEE Trans Inf Technol Biomed 11(3):320–0331

    Article  Google Scholar 

  25. Shao H, Zhang JW, Cui WC, Zhao H (2003) Automatic feature weight assignment based on genetic algorithm for image retrieval. In: 2003 IEEE International Conference on Robotics, Intelligent Systems and Signal Processing. Changsha, China, pp 731–735

  26. Smeulders AWM, Worring M, Santini S, Gupta A, Jain R (2000) Content-based image retrieval at the end of the early years. IEEE Trans Pattern Anal Mach Intell 22(12):1349–1380

    Article  Google Scholar 

  27. Su JH, Huang WJ, Yu PS, Tseng VS (2011) Efficient relevance feedback for content-based image retrieval by mining user navigation patterns. IEEE Trans Knowl Data Eng 23(3):360–372

    Article  Google Scholar 

  28. Tamura H, Mori S, Yamawaki T (1978) Textural features corresponding to visual perception. IEEE Trans Syst Man Cybern 8(6):460–473

    Article  Google Scholar 

  29. Tieu K., Viola P (2004) Boosting image retrieval. Int J Comput Vis 56(1):17–36

    Article  Google Scholar 

  30. Xu X, Lee DJ, Antani S, Long LR (2008) A spine x-ray image retrieval system using partial shape matching. IEEE Trans Inf Technol Biomed 12(1):100–108

    Article  Google Scholar 

  31. Zhang J, Ye L (2009) Content based image retrieval using unclean positive examples. IEEE Trans Image Process 18(10):2370–2375

    Article  MathSciNet  Google Scholar 

  32. Zhang J, Ye L (2009) Image retrieval based on bag of images. In: The 16th IEEE international conference on image processing. Cairo, Egypt, pp 1865–1868

  33. Zhang J, Ye L (2009) Local aggregation function learning based on support vector machines. Signal Process 89(11):2291–2295

    Article  MATH  MathSciNet  Google Scholar 

  34. Zhou XS, Huang TS (2003) Relevance feedback in image retrieval: a comprehensive review. Multimedia Systems 8(6):536–544

    Article  Google Scholar 

Download references

Acknowledgements

The authors thank courtesy of TM Deserno, Dep. of Medical Informatics, RWTH Aachen, Germany, for providing IRMA dataset. This work is supported by the National Natural Science Foundation of China (Multilingual Translation and Integration Using Visual Information for Cross-Language Image Retrieval), the National Natural Science Foundation of China (No. 61108084), the Research Fund for the Doctoral Program of Higher Education of China (Query and Annotation Translation Using Visual Information for Cross-Language Image Retrieval), and the Basic Research Foundation of Beijing Institute of Technology (No. 20120742009).

Author information

Authors and Affiliations

  1. Beijing Engineering Research Center of High Volume Language Information Processing & Cloud Computing Applications, Beijing Institute of Technology, Beijing, 100081, China

    Yonggang Huang & Heyan Huang

  2. School of Computer Science and Technology, Beijing Institute of Technology, Beijing, 100081, China

    Yonggang Huang & Heyan Huang

  3. School of Information Technology, Deakin University, VIC, 3217, Australia

    Jun Zhang

  4. School of Biological Science and Medical Engineering, Beihang University, Beijing, 100191, China

    Daifa Wang

Authors
  1. Yonggang Huang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jun Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Heyan Huang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Daifa Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yonggang Huang.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Huang, Y., Zhang, J., Huang, H. et al. Medical image retrieval based on unclean image bags. Multimed Tools Appl 72, 2977–2999 (2014). https://doi.org/10.1007/s11042-013-1589-3

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-013-1589-3

Keywords

Search

Navigation