skip to main content
10.1145/1999676.1999688acmconferencesArticle/Chapter ViewAbstractPublication Pagesk-capConference Proceedingsconference-collections
research-article

Quality assurance of the content of a large DL-based terminology using mixed lexical and semantic criteria: experience with SNOMED CT

Published: 26 June 2011 Publication History

Abstract

SNOMED-CT is a large medical terminology based on description logic and mandated for use in the US, UK and several other countries. The hierarchies are known to contain many errors, but have so far proved difficult to analyse or quality assure. We present a series of methods and lessons learnt from experience in quality assuring a "module" of SNOMED for specific applications that we expect to generalize both to SNOMED as a whole and to other large ontologies. They feature a) dependence on domain exper-tise b) starting from classes selected for relevance to specific applications, c) tracing all errors to their root and verifying repairs by reclassification d) extraction of manageable-sized "modules"; e) mixed semantic and lexical criteria, and f) extensive use of scripting. They aim to reduce the cognitive load on experts by a) looking initially up-wards rather than downwards in the hierarchies, b) breaking up long lists of direct subclasses by introducing definitions for meaningful subcategories. Errors found range from simple mistakes to systematic errors in schemas.

References

[1]
Andrews, J. E., Richesson, R. L., and Krischer, J. 2007. Variation of SNOMED CT coding of clinical research concepts among coding experts. J American Medical Informatics Association. 14, 4, 497--506.
[2]
Baader, F., Brandt, S., and Lutz, C. 2005. Pushing the EL Envelope. Proc IJCAI-05. 364--369.
[3]
Bodenreider, O., Smith, B., Kumar, A., and Burgun, A. 2007. Investigating subsumption in SNOMED CT: An exploration into large description logic-based biomedical terminologies. Artificial intelligence in medicine. 39, 3, 183--195.
[4]
Campbell, K. E., Tuttle, M. S., and Spackman, K. A. 1998. A "lexically-suggested logical closure" metric for medical terminology maturity. Proc AMIA 1988). 785--789.
[5]
Ceusters, W., Smith, B., Kumar, A., and Dhaen, C. 2004. Mistakes in medical ontologies: Where do they come from and how can they be detected? Studies in Health Technology and Informatics. 145--164.
[6]
Ceusters, W., Smith, B., Kumar, A., and Dhaen, C. 2004. Ontology-based error detection in SNOMED-CT. Proc MEDINFO. 2004, 482--6.
[7]
Chiang, M. F., Hwang, J. C., Yu, A. C., Casper, D. S., and Cimino, J. J. 2006. Reliability of SNOMED-CT Coding by Three Physicians using Two Terminology Browsers. Proc AMIA 2006. 131--135.
[8]
Elkin, P. L., Ruggieri, A. P., Brown, S. H., Buntrock, J., Bauer, B. A., Wahner-Roedler, D., Litin, S. C., Beinborn, J., Bailey, K. R., and Bergstrom, L. 2001. A randomized controlled trial of the accuracy of clinical record retrieval using SNOMED-RT as compared with ICD9-CM. Proc AMIA 2001. 159--164.
[9]
Fung, K. W., McDonald, C., and Srinivasan, S. 2010. The UMLS-CORE project: a study of the problem list terminologies used in large healthcare institutions. J Am Med Inform Assoc. 17, 675--680.
[10]
Goldbeck, J., Fragoso, G., Hartel, F., Hendler, J., Oberthaler, J., and Parsia, B. 2004. The National Can-cer Institute's thesaurus and ontology. J Web Semantics. 1, 1, 32--36.
[11]
Grau, B. C., Horrocks, I., Kazakov, Y., and Sattler, U. 2008. Modular reuse of ontologies: Theory and practice. J Artificial Intelligence Research. 31, 1, 273--318.
[12]
Grice, H. P. 1957. Meaning. Phil Rev. 66, 377--388.
[13]
Horridge, M., Parsia, B., and Sattler, U. 2010. Justification oriented proofs in OWL. International Semantic Web Conference (ISWC 2010). 354--369.
[14]
Iannone, L., Aranguren, M. E., Rector, A., and Stevens, R. 2008. Augmenting the expressivity of the ontology pre-processor language. OWL Experiences and Directions (OWLEd 2008).
[15]
Iannone, L., Rector, A., and R, S. 2009. Embedding knowledge patterns into OWL. European Semantic Web Conference (ESWC 2009). 218--232.
[16]
Lawley, M. J. Exploiting fast classification of SNOMED CT for query and integration of health data. KR-MED 2008. 8--14.
[17]
Rector, A. and Brandt, S. 2008. Why do it the hard way? The case for an expressive ontological schemas for SNOMED. J Am Med Inform Assoc. 15, 744--751.
[18]
Rector, A., Brandt, S., and Schneider, T. 2011. Getting the foot out of the pelvis: Modelling problems affecting use of SNOMED-CT hierarchies in practical applications. JAMIA 18, (in press).
[19]
Rector, A. 2003. Modularisation of domain ontologies Implemented in description logics and related formalisms including OWL. Proc KCAP 2003. 121--128.
[20]
Rogers, J., Roberts, A., Solomon, D., van der Haring, E., Wroe, C., Zanstra, P., and Rector, A. 2001. GALEN Ten years on: Tasks and supporting tools. Proc Medinfo 2001. 256--260.
[21]
Schulz, S., Suntisrivaraporn, B., and Baader, F. 2007. SNOMED CT's Problem List: Ontologists' and logicians' therapy suggestions. Proc Medinfo 2007. 802--806.
[22]
Schulz, S., Suntisrivaraporn, B., Baader, F., and Boeker, M. 2009. SNOMED reaching its adolescence: Ontologists' and logicians' health check. International Journal of Medical Informatics. 78, S86-S94.
[23]
Schulz, S., Hahn, U., and Romacker, M. 2000. Modeling anatomical spatial relations with description logics. AMIA Fall Symposium (AMIA-2000). 799--783.
[24]
Spackman, K. A. and Reynoso, G. 2004. Examining SNOMED from the perspective of formal ontological principles: Some preliminary analysis and observations. KR-MED. 81--87.
[25]
Vikström, A., Skånér, Y., Strender, L. E., and Nilsson, G. H. 2007. Mapping the categories of the Swedish primary health care version of ICD-10 to SNOMED CT concepts: Rule development and intercoder reliability in a mapping trial. BMC Medical Informatics and Decision Making. 7, 1, 9.
[26]
Wang, A. Y., Sable, J. H., and Spackman, K. A. 2002. The SNOMED clinical terms development process: re-finement and analysis of content. AMIA Fall Symposium. 845.
[27]
Wang, Y., Halper, M., Min, H., Perl, Y., Chen, Y., and Spackman, K. A. 2007. Structural methodologies for auditing SNOMED. J Biomed Informatics. 40, 5, 561--581.

Cited By

View all
  • (2018)From lexical regularities to axiomatic patterns for the quality assurance of biomedical terminologies and ontologiesJournal of Biomedical Informatics10.1016/j.jbi.2018.06.00884(59-74)Online publication date: Aug-2018
  • (2014)Formal Representations and Semantic Web TechnologiesClinical Decision Support10.1016/B978-0-12-398476-0.00020-8(551-598)Online publication date: 2014

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
K-CAP '11: Proceedings of the sixth international conference on Knowledge capture
June 2011
212 pages
ISBN:9781450303965
DOI:10.1145/1999676
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 26 June 2011

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. OWL
  2. description logics
  3. modularization
  4. ontologies
  5. quality assurance

Qualifiers

  • Research-article

Conference

K-CAP '2011
Sponsor:
K-CAP '2011: Knowledge Capture Conference
June 26 - 29, 2011
Alberta, Banff, Canada

Acceptance Rates

Overall Acceptance Rate 55 of 198 submissions, 28%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)3
  • Downloads (Last 6 weeks)0
Reflects downloads up to 17 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2018)From lexical regularities to axiomatic patterns for the quality assurance of biomedical terminologies and ontologiesJournal of Biomedical Informatics10.1016/j.jbi.2018.06.00884(59-74)Online publication date: Aug-2018
  • (2014)Formal Representations and Semantic Web TechnologiesClinical Decision Support10.1016/B978-0-12-398476-0.00020-8(551-598)Online publication date: 2014

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media