Skip to main content

Advertisement

SpringerLink
Log in

Semantic-Enhanced Query Expansion System for Retrieving Medical Image Notes

  • Systems-Level Quality Improvement
  • Published:
Journal of Medical Systems Aims and scope Submit manuscript

Abstract

Most current image retrieval methods require constructing semantic metadata for representing image content. To manually create semantic metadata for medical images is time-consuming, yet it is a crucial component for query expansion. We proposed a new method for searching medical image notes that uses semantic metadata to improve query expansion and leverages a knowledge model developed specifically for the medical image domain to create relevant metadata. We used a syntactic parser and the Unified Medical Language System to analyze the corpus and store text information as semantic metadata in a knowledge model. Our new method has an interactive interface that allows users to provide relevance feedback and construct new queries more efficiently. Sixteen medical professionals evaluated the query expansion module, and each evaluator had prior experience searching for medical images. When using the initial query as the baseline standard, expanded queries achieved a performance boost of 22.6% in terms of the relevance score on first ten results (P-value<0.05). When using Google as another baseline, our system performed 24.6% better in terms of relevance score on the first ten results (P-value<0.05). Overall, 75% of the evaluators said the semantic-enhanced query expansion workflow is logical, easy to follow, and comfortable to use. In addition, 62% of the evaluators preferred using our system instead of Google. Evaluators who were positive about our system found the knowledge map-based visualization of candidate medical search terms helpful in refining cases from the initial search results.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

References

  1. Shatkay, H., Chen, N., and Blostein, D., Integrating image data into biomedical text categorization. Bioinformatics 22(14):e446–e453, 2006.

    Article  PubMed  CAS  Google Scholar 

  2. Chen, T., Lu, D., Kan, M.-Y., and Cui, P. Understanding and classifying image tweets. In: Proceedings of the 21st ACM international conference on Multimedia. ACM, 2013. pp 781–784

  3. Xu, S., McCusker, J., and Krauthammer, M., Yale image finder (YIF): A new search engine for retrieving biomedical images. Bioinformatics 24(17):1968–1970, 2008.

    Article  PubMed  PubMed Central  CAS  Google Scholar 

  4. Rubin, D. L., Mongkolwat, P., Kleper, V., and Supekar, K. Channin DS medical imaging on the semantic web: annotation and image markup. In: AAAI Spring Symposium: Semantic Scientific Knowledge Integration. 2008. pp 93–98.

  5. Jeon, J., Lavrenko, V., and Manmatha, R. Automatic image annotation and retrieval using cross-media relevance models. In: Proceedings of the 26th annual international ACM SIGIR conference on Research and development in informaion retrieval. ACM, 2003. pp 119–126.

  6. Liu, Z., and Chu, W. W., Knowledge-based query expansion to support scenario-specific retrieval of medical free text. Inf. Retr. 10(2):173–202, 2007.

    Article  Google Scholar 

  7. Zenz, G., Zhou, X., Minack, E., Siberski, W., and Nejdl, W., From keywords to semantic queries—Incremental query construction on the semantic web. Web Semant. Sci. Serv. Agents World Wide Web 7(3):166–176, 2009.

    Article  Google Scholar 

  8. Russell, A., Smart, P. R., Braines, D., and Shadbolt, N. R. (2008) Nitelight: a graphical tool for semantic query construction.

  9. Kahn, C. E., and Rubin, D. L., Automated semantic indexing of figure captions to improve radiology image retrieval. J. Am. Med. Inform. Assoc. 16(3):380–386, 2009.

    Article  PubMed  PubMed Central  Google Scholar 

  10. Philbin, J., Chum, O., Isard, M., Sivic, J., and Zisserman, A. Object retrieval with large vocabularies and fast spatial matching. In: Computer Vision and Pattern Recognition, 2007. CVPR'07. IEEE Conference on. IEEE, 2007. pp 1–8

  11. Philbin, J., Chum, O., Isard, M., Sivic, J, and Zisserman, A. Lost in quantization: Improving particular object retrieval in large scale image databases. In: Computer Vision and Pattern Recognition, 2008. CVPR 2008. IEEE Conference on. IEEE, 2008. pp 1–8

  12. Xu, J., and Croft, W. B. Query expansion using local and global document analysis. In: Proceedings of the 19th annual international ACM SIGIR conference on Research and development in information retrieval. ACM, 1996. pp 4–11.

  13. Cao, G., Nie, J.-Y., Gao, J., and Robertson, S. Selecting good expansion terms for pseudo-relevance feedback. In: Proceedings of the 31st annual international ACM SIGIR conference on Research and development in information retrieval. ACM, 2008. pp 243–250.

  14. Natsev, A. P., Haubold, A., Tešić, J., Xie, L., and Yan, R. Semantic concept-based query expansion and re-ranking for multimedia retrieval. In: Proceedings of the 15th ACM international conference on Multimedia. ACM, 2007. pp 991–1000.

  15. Chu, W. W., Liu, Z., and Mao, W. Textual document indexing and retrieval via knowledge sources and data mining. Communication of the Institute of Information and Computing Machinery (CIICM), Taiwan 5(2), 2002.

  16. Tong S, Chang E Support vector machine active learning for image retrieval. In: Proceedings of the ninth ACM international conference on Multimedia. ACM, 2001, pp 107–118

  17. Attar, R., and Fraenkel, A. S., Local feedback in full-text retrieval systems. J ACM 24(3):397–417, 1977.

    Article  Google Scholar 

  18. Minker, J., Wilson, G. A., and Zimmerman, B. H., An evaluation of query expansion by the addition of clustered terms for a document retrieval system. Inf. Storage Retr. 8(6):329–348, 1972.

    Article  Google Scholar 

  19. White, R. W., Kules, B., and Drucker, S. M., Supporting exploratory search, introduction, special issue, communications of the ACM. Commun. ACM 49(4):36–39, 2006.

    Article  Google Scholar 

  20. Hearst, M. A., Clustering versus faceted categories for information exploration. Commun. ACM 49(4):59–61, 2006.

    Article  Google Scholar 

  21. Zhao, Y., Fesharaki, N. J., Liu, H., and Luo, J. Using data-driven sublanguage pattern mining to induce knowledge models: application in medical image reports knowledge representation. Submitted to BMC Medical Informatics and Decision Making

  22. Radiopaedia. http://radiopaedia.org/.

  23. Apache Lucene. https://lucene.apache.org/.

  24. D3.js. https://d3js.org/.

Download references

Acknowledgements

This work was made possible by the UWM Foundation and GE Healthcare Catalyst Fund, and the Center for Biomedical Data and Language Processing in collaboration with Department of Health Informatics and Administration in the College of Health Sciences at the University of Wisconsin-Milwaukee. Its contents are solely the responsibility of the authors and do not necessarily represent the official views of the funder.

Author information

Authors and Affiliations

  1. College of Health Sciences, Center for Biomedical Data and Language Processing, University of Wisconsin-Milwaukee, Milwaukee, WI, USA

    Yiqing Zhao, Nooshin J. Fesharaki, Timothy B. Patrick & Jake Luo

  2. ChengDe Petroleum College, Chengde, China

    Xiaohui Li

Authors
  1. Yiqing Zhao
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Nooshin J. Fesharaki
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Xiaohui Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Timothy B. Patrick
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Jake Luo
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jake Luo.

Ethics declarations

Conflict of Interest

The authors report no financial interests or potential conflicts of interest.

Data Depository

Data used in the manuscript is from an online data repository: Radiopaedia.org.

Additional information

This article is part of the Topical Collection on Systems-Level Quality Improvement

Appendix

Appendix

Table 2 Regrouping of UMLS semantic types to form 14 semantic categories (four conceptually important semantic types highlighted in red) [21]
Full size table

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhao, Y., Fesharaki, N.J., Li, X. et al. Semantic-Enhanced Query Expansion System for Retrieving Medical Image Notes. J Med Syst 42, 105 (2018). https://doi.org/10.1007/s10916-018-0954-1

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s10916-018-0954-1

Keywords

Search

Navigation