Skip to main content

Advertisement

SpringerLink
Log in

Task-based annotation and retrieval for image information management

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Continuing advances in digital image capture and storage are resulting in a proliferation of imagery and associated problems of information overload in image domains. In this work we present a framework that supports image management using an interactive approach that captures and reuses task-based contextual information. Our framework models the relationship between images and domain tasks they support by monitoring the interactive manipulation and annotation of task-relevant imagery. During image analysis, interactions are captured and a task context is dynamically constructed so that human expertise, proficiency and knowledge can be leveraged to support other users in carrying out similar domain tasks using case-based reasoning techniques. In this article we present our framework for capturing task context and describe how we have implemented the framework as two image retrieval applications in the geo-spatial and medical domains. We present an evaluation that tests the efficiency of our algorithms for retrieving image context information and the effectiveness of the framework for carrying out goal-directed image tasks.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14

Similar content being viewed by others

References

  1. Allen J (2002) Challenges in information retrieval and language modelling: report of a workshop held at the centre for intelligent information retrieval, University of Massachusetts. SIGIR Forum 37:31–47

    Article  Google Scholar 

  2. Althoff K, Webber R (2006) Knowledge management in case-based reasoning. Knowledge Eng Rev 20:305–310

    Article  Google Scholar 

  3. Andriole KP, Morin RL, Arenson RL, Carrino JA, Erickson BJ, Horii SC, Piraino DW, Reiner BI, Seibert JA, Siegel E (2004) Addressing the coming radiology crisis: the society for computer applications in radiology transforming the radiological interpretation process (trip) initiative. J Digit Imaging 17:235–243

    Article  Google Scholar 

  4. Belkin NJ, Callan J (2003) Context-based information access. Report of the Discussion Group on Context-Based Information Access of the Workshop on Information Retrieval and Databases: Synergies and Syntheses National Science Foundation. http://www2.cs.washington.edu/nsf2003/discussionGroups.html

  5. Bichindaritz I (2003) Solving safety implications in a case-based decision-support system in medicine. Proceedings of the Fifth International Conference on Case-based Reasoning Workshop on Case-Based Reasoning in the Health Sciences, pp 178–183

  6. Bichindaritz I (2008) Prototypical case mining from biomedical literature for bootstrapping a case base. Appl Intell 28:222–237

    Article  Google Scholar 

  7. Bichindaritz I, Marling C (2006) Case-based reasoning in the health sciences: what’s next? Artif Intell Med 36:127–135

    Article  Google Scholar 

  8. Bradley F, Jung B (2005) Putting fun into function with QuizMed—an interactive medical application. Proceedings of the Eighteenth International Conference on Computer Based Medical Systems, pp 226–231

  9. Budzik J, Hammond KJ (2000) User interactions with everyday applications as context for just-in-time information access. Proceedings of the Fifth International Conference on Intelligent User Interfaces pp 44–51

  10. Budzik J, McLoughlin L, Hammond K (2003) Information access in context: experiences with the Watson system. Dissertation, Northwestern University

  11. Burke R, Kass A (2000) Retrieving stories for case-based teaching. In: Leake D (ed) Case-based reasoning: experiences, lessons, and future directions, 2nd edn. AAAI/MIT, pp 93–109

  12. Claypool M, Le P, Waseda M, Brown D (2001) Implicit interest indicators. Proceedings of the Sixth International Conference on Intelligent User Interfaces, pp 33–40

  13. Demner-Fushman D, Antani S, Simpson M, Thoma G (2009) Annotation and retrieval of clinically relevant images. Int J MedInform 78:59–67

    Google Scholar 

  14. DermAtlas: Online Dermatology Image Library (2009) Accessed 10 Apr. Available from http://dermatlas.med.jhmi.edu/derm/

  15. Enser PGB, Sandom CJ, Lewis PH (2005) Automatic annotation of images from the practitioner perspective. Proceedings of the 4th International Conference on Image and Video Retrieval, pp 497–506

  16. Fan J, Gao Y, Luo H (2004) Multi-level annotation of natural scenes using dominant image components and semantic concepts. Proceedings of the ACM International Conference on Multimedia, pp 540–547

  17. Flickner M, Sawhney H, Ashley J, Huang Q, Dom B, Gorkani M, Hafner J, Lee D, Petkovic D, Steele D, Yanker P (1995) Query by image and video content: the qbic system. IEEE Comput 28:23–32

    Google Scholar 

  18. Frucci M, Perner P, di Baja GS (2008) Case-based-reasoning for image segmentation. Int J Pattern Recognit Artif Intell 22:829–842

    Article  Google Scholar 

  19. Gandhi V, Kang JM, Shekhar S (2009) In: Spatial databases: encyclopaedia of computer science and engineering. Wiley, New York

  20. Grimnes M, Aamodt A (1996) A two layer case-based reasoning architecture for medical image understanding. Proceedings of the Second European Workshop on Case-based Reasoning, pp 164–178

  21. Hare JS, Lewis PH, Enser PGB, Sandom CJ (2006) Mind the gap: another look at the problem of the semantic gap in image retrieval. In: Chang EY, Hanjalic A, Sebe N (eds) Multimedia Content Analysis, Management, and Retrieval 6073:1–12

  22. Holt A, Bichindaritz I, Schmidt R, Perner P (2006) Medical applications in case-based reasoning. Knowledge Eng Rev 20:289–292

    Article  Google Scholar 

  23. Hussain F, Abidi SSR (2005) A knowledge management framework to morph clinical cases with clinical practice guidelines. Stud Health Technol Inform 116:731–736

    Google Scholar 

  24. Kelly D, Teevan J (2003) Implicit feedback for inferring user preference: a bibliography. SIGIR Forum 37:18–28

    Article  MATH  Google Scholar 

  25. Lewis L, Foxx L (2005) NASA Takes Google on Journey into Space. Accessed 6 Jul 2009. Available from http://www.nasa.gov/centers/ames/news/releases/2005/05_50AR.html

  26. Lieberman H (1995) Letizia: An agent that assists web browsing. Proceedings of the Fourteenth International Joint Conference on Artificial Intelligence, pp 924–929

  27. Marling C, Whitehouse P (2001) Case-based reasoning in the care of Alzheimer’s disease patients. Proceedings of the Fourth International Conference on Case-Based Reasoning, pp 702–715

  28. Minor M (2006) Experience management with case-based assistant systems. Proceedings of the Eighth European Conference on Case-Based Reasoning, pp 182–195

  29. Müller H, Kalpathy-Cramer J, Kahn CE Jr, Hatt W, Bedrick S, Hersh W (2008) Overview of the ImageCLEFmed 2008 medical image retrieval task. Available from: http://www.clef-campaign.org/2008/working%5Fnotes/

  30. Nilsson M, Sollenborn M (2004) Advancements and trends in medical case-based reasoning: An overview of systems and system development. Proceedings of the Seventeenth International Florida Artificial Intelligence Research Society Conference, pp 178–183

  31. O’Sullivan D, Smyth B, Wilson D (2003) Explicit vs. implicit profiling: a case-study in electronic programme guides. Proceedings of the Eighteenth International Joint Conference on Artificial Intelligence, pp 1351–1359

  32. Perner P, Holt A, Richter M (2006) Image processing in case-based reasoning. Knowledge Eng Rev 20:311–314

    Article  Google Scholar 

  33. Rhodes B (2003) Using physical context for just-in-time information retrieval. IEEE Trans Comput 52:1011–1014

    Article  Google Scholar 

  34. Salton G, McGill M (1983) Introduction to modern information retrieval. McGraw-Hill.

  35. Solomon P (2003) Looking for information—a survey of research on information seeking, needs, and behaviour. Inf Retrieval 6:284–288

    Article  Google Scholar 

  36. The Apache Lucene (2008) The Apache Software Foundation; Accessed 30 Oct. Available from http://lucene.apache.org/

  37. Uchihashi S, Kanade T (2005) Content-free image retrieval based on relations exploited from user feedbacks. Proceedings of IEEE International Conference on Multimedia and Expo, pp 1358–1361

  38. Wang J, Li J (2003) Automatic linguistic indexing of pictures by a statistical modelling approach. IEEE T Pattern Ana 25:1075–1088

    Article  Google Scholar 

  39. Weber R, Aha DW, Branting K, Lucas JR, Becerra-Fernandez I (2000) Active case-based reasoning for lessons delivery system. Proceedings of the Thirtieth Florida Artificial Intelligence Research Society Conference, pp 170–174

  40. Worring M, Schrieber G (2007) Semantic image and video indexing in broad domains. IEEE Trans Multimedia 9:909–919

    Article  Google Scholar 

  41. Zhao R, Grosky WI (2000) Negotiating the semantic gap: from feature maps to semantic landscapes. Pattern Recogn 35:593–600

    Article  Google Scholar 

Download references

Acknowledgements

The support of the Proof of Concept and Commercialization Funds of Enterprise Ireland is gratefully acknowledged.

Author information

Authors and Affiliations

  1. School of Engineering and Applied Science, University of Aston, Birmingham, UK

    Dympna O’Sullivan

  2. Department of Software and Information Systems, University of North Carolina, Charlotte, NC, USA

    David C. Wilson

  3. School of Computer Science and Informatics, University College Dublin, Dublin, Ireland

    Michela Bertolotto

Authors
  1. Dympna O’Sullivan
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. David C. Wilson
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Michela Bertolotto
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Dympna O’Sullivan.

Rights and permissions

Reprints and permissions

About this article

Cite this article

O’Sullivan, D., Wilson, D.C. & Bertolotto, M. Task-based annotation and retrieval for image information management. Multimed Tools Appl 54, 473–497 (2011). https://doi.org/10.1007/s11042-010-0548-5

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-010-0548-5

Keywords

Search

Navigation