Skip to main content
SpringerLink
Log in

Scalable Medical Image Understanding by Fusing Cross-Modal Object Recognition with Formal Domain Semantics

  • Conference paper
Biomedical Engineering Systems and Technologies (BIOSTEC 2008)

Part of the book series: Communications in Computer and Information Science ((CCIS,volume 25))

Abstract

Recent advances in medical imaging technology have dramatically increased the amount of clinical image data. In contrast, techniques for efficiently exploiting the rich semantic information in medical images have evolved much slower. Despite the research outcomes in image understanding, current image databases are still indexed by manually assigned subjective keywords instead of the semantics of the images. Indeed, most current content-based image search applications index image features that do not generalize well and use inflexible queries. This slow progress is due to the lack of scalable and generic information representation systems which can abstract over the high dimensional nature of medical images as well as semantically model the results of object recognition techniques. We propose a system combining medical imaging information with ontological formalized semantic knowledge that provides a basis for building universal knowledge repositories and gives clinicians fully cross-lingual and cross-modal access to biomedical information.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 129.00
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 169.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Berners-Lee, T., Hendler, J., Lassila, O.: The semantic web – a new form of web content that is meaningful to computers will unleash a revolution of new possibilities. Scientific American (2001)

    Google Scholar 

  2. Brickley, D., Guha, R.: RDF vocabulary description language 1.0: RDF Schema (2004)

    Google Scholar 

  3. McGuinness, D.L., van Harmelen, F.: OWL web ontology language overview (2004)

    Google Scholar 

  4. Hong, W., Georgescu, B., Zhou, X.S., Krishnan, S., Ma, Y., Comaniciu, D.: Database-guided simultaneous multi-slice 3D segmentation for volumetric data. In: Leonardis, A., Bischof, H., Pinz, A. (eds.) ECCV 2006. LNCS, vol. 3954, pp. 397–409. Springer, Heidelberg (2006)

    Chapter  Google Scholar 

  5. Tu, Z., Zhou, X.S., Bogoni, L., Barbu, A., Comaniciu, D.: Probabilistic 3D polyp detection in CT images: The role of sample alignment. IEEE Computer Society Conference on Computer Vision and Pattern Recognition 2, 1544–1551 (2006)

    Google Scholar 

  6. Comaniciu, D., Zhou, X., Krishnan, S.: Robust real-time myocardial border tracking for echocardiography: an information fusion approach. IEEE Transactions in Medical Imaging 23(7), 849–860 (2004)

    Article  Google Scholar 

  7. Chan, A.B., Moreno, P.J., Vasconcelos, N.: Using statistics to search and annotate pictures: an evaluation of semantic image annotation and retrieval on large databases. In: Proceedings of Joint Statistical Meetings (JSM) (2006)

    Google Scholar 

  8. Müller, H., Deselaers, T., Lehmann, T., Clough, P., Kim, E., Hersh, W.: Overview of the ImageCLEFmed 2006 medical retrieval and annotation tasks. In: Peters, C., Gey, F.C., Gonzalo, J., Müller, H., Jones, G.J.F., Kluck, M., Magnini, B., de Rijke, M., Giampiccolo, D. (eds.) CLEF 2005. LNCS, vol. 4022, Springer, Heidelberg (2006)

    Chapter  Google Scholar 

  9. Gruber, T.R.: Toward principles for the design of ontologies used for knowledge sharing. International Journal of Human-Computer Studies 43, 907–928 (1995)

    Article  Google Scholar 

  10. Rosse, C., Mejino, R.L.V.: A reference ontology for bioinformatics: The Foundational Model of Anatomy. Journal of Biomedical Informatics 36, 478–500 (2003)

    Article  PubMed  Google Scholar 

  11. Buitelaar, P., Sintek, M., Kiesel, M.: A lexicon model for multilingual/multimedia ontologies. In: Sure, Y., Domingue, J. (eds.) ESWC 2006. LNCS, vol. 4011. Springer, Heidelberg (2006)

    Google Scholar 

  12. Volk, M., Ripplinger, B., Vintar, S., Buitelaar, P., Raileanu, D., Sacaleanu, B.: Semantic annotation for concept-based cross-language medical information retrieval. International Journal of Medical Informatics (2003)

    Google Scholar 

  13. Vintar, S., Buitelaar, P., Volk, M.: Semantic relations in concept-based cross-language medical information retrieval. In: Proceedings of the ECML/PKDD Workshop on Adaptive Text Extraction and Mining (ATEM ) (2003)

    Google Scholar 

  14. Carbonell, J.G., Yang, Y., Frederking, R.E., Brown, R.D., Geng, Y., Lee, D.: Translingual information retrieval: A comparative evaluation. International Journal of Computational Intelligence and Applications 1, 708–715 (1997)

    Google Scholar 

  15. Eichmann, D., Ruiz, M.E., Srinivasan, P.: Cross-language information retrieval with the UMLS metathesaurus. In: Proceedings of the 21st annual international ACM SIGIR conference on Research and development in information retrieval, pp. 72–80 (1998)

    Google Scholar 

  16. Laender, A., Ribeiro-Neto, B., Silva, A., Teixeira, J.: A brief survey of web data extraction tools. Special Interest Group on Management of Data (SIGMOD) Record 31(2) (2002)

    Google Scholar 

  17. Ravichandran, D., Hovy, E.: Learning surface text for a question answering system. In: Proceedings of the Association of Computational Linguistics Conference (ACL) (2002)

    Google Scholar 

  18. Ling, C., Gao, J., Zhang, H., Qian, W., Zhang, H.: Improving encarta search engine performance by mining user logs. International Journal of Pattern Recognition and Artificial Intelligence 16(8) (2002)

    Google Scholar 

  19. Deselaers, T., Weyand, T., Keysers, D., Macherey, W., Ney, H.: FIRE in ImageCLEF 2005: Combining content-based image retrieval with textual information retrieval. In: Working Notes of the CLEF Workshop (2005)

    Google Scholar 

  20. Hunter, J.: Adding multimedia to the semantic web - building an MPEG-7 ontology. In: International Semantic Web Working Symposium (SWWS) (2001)

    Google Scholar 

  21. Barnard, K., Duygulu, P., Forsyth, D., de Freitas, N., Blei, D., Jordan, M.: Matching words and pictures. Journal of Machine Learning Research 3, 1107–1135 (2003)

    Google Scholar 

  22. Lavrenko, V., Manmatha, R., Jeon, J.: A model for learning the semantics of pictures. In: Proceedings of the Neural Information Processing Systems Conference (2003)

    Google Scholar 

  23. Lim, J.H.: Learnable visual keywords for image classification. In: DL 1999: Proceedings of the fourth ACM conference on Digital libraries, pp. 139–145. ACM Press, New York (1999)

    Chapter  Google Scholar 

  24. Carneiro, G., Vasconcelos, N.: A database centric view of semantic image annotation and retrieval. In: SIGIR 2005: Proceedings of the 28th annual international ACM SIGIR conference on Research and development in information retrieval, pp. 559–566 (2005)

    Google Scholar 

  25. Town, C.: Ontological inference for image and video analysis. International Journal of Machine Vision and Applications 17, 94–115 (2006)

    Article  Google Scholar 

  26. Mezaris, V., Kompatsiaris, I., Strintzis, M.: An ontology approach to object-based image retrieval. In: International Conference on Image Processing (ICIP) (2003)

    Google Scholar 

  27. Mojsilovic, A., Gomes, J., Rogowitz, B.E.: Semantic-friendly indexing and querying of images based on the extraction of the objective semantic cues. International Journal of Computer Vision 56, 79–107 (2004)

    Article  Google Scholar 

  28. Lehmann, T., Gld, M., Thies, C., Fischer, B., Spitzer, K., Keysers, D., Ney, H., Kohnen, M., Schubert, H., Wein, B.: The IRMA project. A state of the art report on content-based image retrieval in medical applications. In: Proceedings 7th Korea-Germany Joint Workshop on Advanced Medical Image Processing, pp. 161–171 (2003)

    Google Scholar 

  29. Vompras, J.: Towards adaptive ontology-based image retrieval. In: Stefan Brass, C.G. (ed.) 17th GI-Workshop on the Foundations of Databases, Wörlitz, Germany, Institute of Computer Science, Martin-Luther-University Halle-Wittenberg, pp. 148–152 (2005)

    Google Scholar 

  30. Papadopoulosa, G.T., Mezaris, V., Dasiopoulou, S., Kompatsiaris, I.: Semantic image analysis using a learning approach and spatial context. In: Proceedings of the 1st international conference on Semantics And digital Media Technologies (SAMT) (2006)

    Google Scholar 

  31. Su, L., Sharp, B., Chibelushi, C.: Knowledge-based image understanding: A rule-based production system for X-ray segmentation. In: Proceedings of Fourth International Conference on Enterprise Information System, Ciudad Real, Spain, vol. 1, pp. 530–533 (2002)

    Google Scholar 

  32. Mechouche, A., Golbreich, C., Gibaud, B.: Towards a hybrid system using an ontology enriched by rules for the semantic annotation of brain MRI images. In: Marchiori, M., Pan, J.Z., de Sainte Marie, C. (eds.) RR 2007. LNCS, vol. 4524, pp. 219–228. Springer, Heidelberg (2007)

    Chapter  Google Scholar 

  33. Horrocks, I., Patel-Schneider, P.F., Boley, H., Tabet, S., Grosof, B., Dean, M.: SWRL: A semantic web rule language combining OWL and RuleML. Technical report, W3C Member submission, May 21 (2004)

    Google Scholar 

  34. Castano, S., Ferrara, A., Hess, G.: Discovery-driven ontology evolution. In: SWAP 2006: Semantic Web Applications and Perspectives - 3rd Italian Semantic Web Workshop (2006)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. DFKI GmbH, Knowledge Management Dept, Kaiserslautern, Germany

    Manuel Möller & Michael Sintek

  2. DFKI GmbH, Language Technology Lab, Saarbrücken, Germany

    Paul Buitelaar

  3. Siemens Corporate Research, 755 College Road East, Princeton NJ, U.S.A.

    Saikat Mukherjee

  4. Siemens Medical Solutions, 51 Valley Stream Parkway, Malvern PA, U.S.A.

    Xiang Sean Zhou

  5. Siemens Medical Solutions, Hartmannstr. 16, 91052, Erlangen, Germany

    Jörg Freund

Authors
  1. Manuel Möller
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Michael Sintek
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Paul Buitelaar
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Saikat Mukherjee
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Xiang Sean Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Jörg Freund
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. IST - Instituto Superior Técnico, Instituto de Telecomunicaçöes, Av. Rovisco Pais, 1, 1049-001, Lisbon, Portugal

    Ana Fred

  2. Polytechnic Institute of Setúbal – INSTICC, Departament of Systems and Informatics, Rua do Vale de Chaves - Estefanilha, 2910-761, Setúbal, Portugal

    Joaquim Filipe

  3. Institute of Telecommunications, Av. Rovisco Pais, 1, 1049-001, Lisboa, Portugal

    Hugo Gamboa

Rights and permissions

Reprints and permissions

Copyright information

© 2008 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Möller, M., Sintek, M., Buitelaar, P., Mukherjee, S., Zhou, X.S., Freund, J. (2008). Scalable Medical Image Understanding by Fusing Cross-Modal Object Recognition with Formal Domain Semantics. In: Fred, A., Filipe, J., Gamboa, H. (eds) Biomedical Engineering Systems and Technologies. BIOSTEC 2008. Communications in Computer and Information Science, vol 25. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-92219-3_29

Download citation

  • DOI: https://doi.org/10.1007/978-3-540-92219-3_29

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-540-92218-6

  • Online ISBN: 978-3-540-92219-3

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation