skip to main content
research-article

Conformance Checking Methodology Across Discharge Summaries and Standard Treatment Guidelines

Published: 30 May 2020 Publication History

Abstract

Conformance checking of treatment plans in discharge summary data would facilitate the development of clinical decision support system, treatment plan quality assurance, and new treatment plan discovery. Conformance checking requires extraction of medical entities and relationships among them to form a computable representation of the treatment plan present in the discharge summary. We propose a workflow representation of patient’s discharge summary that is referred to as workflow instance. We employ a multi-layer perceptron neural network to extract relationships between medical entities to construct the workflow instance. The aim of this work is to check the conformance of the workflow instance against standard treatment plan. Standard treatment plans are extracted from the treatment guidelines provided on healthcare websites such as WebMD, Mayo Clinic, and Johns Hopkins. For each disease, these guidelines are curated, aggregated, and represented as a workflow specification. We commend multiple measures to compute the conformance of workflow instance with workflow specification. We validate our conformance checking methodology using discharge summary data of three diseases, namely colon cancer, coronary artery disease, and brain tumor, collected from THYME corpus and MIMIC III clinical database. Our approach and the solution can be used by hospitals and patients to determine adherence, gaps, and additions to standard treatment plans. Further, our work can facilitate to identify common errors and goodness in actual enactment of treatment plans, which can further lead to refinement of standard treatment plans.

References

[1]
Asma Ben Abacha and Pierre Zweigenbaum. 2011. Automatic extraction of semantic relations between medical entities: A rule based approach. Journal of Biomedical Semantics 2, 5 (2011), S4.
[2]
Sophia Ananiadou, Sampo Pyysalo, Jun’ichi Tsujii, and Douglas B. Kell. 2010. Event extraction for systems biology by text mining the literature. Trends in Biotechnology 28, 7 (2010), 381--390.
[3]
Susan E. Andrade, Jerry H. Gurwitz, Terry S. Field, Michael Kelleher, Sumit R. Majumdar, George Reed, and Robert Black. 2004. Hypertension management: The care gap between clinical guidelines and clinical practice. American Journal of Managed Care 10, 7 Pt 2 (2004), 481--486.
[4]
Alan R. Aronson. 2006. MetaMap: Mapping Text to the UMLS® Metathesaurus®. Bethesda, MD: NLM, NIH, DHHS.
[5]
Iyad Batal, Hamed Valizadegan, Gregory F. Cooper, and Milos Hauskrecht. 2013. A temporal pattern mining approach for classifying electronic health record data. ACM Transactions on Intelligent Systems and Technology 4, 4 (2013), 63.
[6]
L. Bessette, L.-G. Ste-Marie, S. Jean, K. S. Davison, M. Beaulieu, M. Baranci, J. Bessant, and J. P. Brown. 2008. The care gap in diagnosis and treatment of women with a fragility fracture. Osteoporosis International 19, 1 (2008), 79--86.
[7]
Olivier Bodenreider. 2004. The Unified Medical Language System (UMLS): Integrating biomedical terminology. Nucleic Acids Research 32, Suppl. 1 (2004), D267--D270.
[8]
Horst Bunke and Kim Shearer. 1998. A graph distance metric based on the maximal common subgraph. Pattern Recognition Letters 19, 3--4 (1998), 255--259.
[9]
Chih-Chung Chang and Chih-Jen Lin. 2011. LIBSVM: A library for support vector machines. ACM Transactions on Intelligent Systems and Technology 2, 3 (2011), 27.
[10]
Ronan Collobert and Samy Bengio. 2004. Links between perceptrons, MLPs and SVMs. In Proceedings of the 21st International Conference on Machine Learning. ACM, New York, NY, 23.
[11]
Berry de Bruijn, Colin Cherry, Svetlana Kiritchenko, Joel Martin, and Xiaodan Zhu. 2010. NRC at i2b2: One challenge, three practical tasks, nine statistical systems, hundreds of clinical records, millions of useful features. In Proceedings of the 2010 i2b2/VA Workshop on Challenges in Natural Language Processing for Clinical Data.
[12]
Marc Ehrig, Agnes Koschmider, and Andreas Oberweis. 2007. Measuring similarity between semantic business process models. In Proceedings of the 4th Asia-Pacific Conference on Conceptual Modelling, Volume 67. 71--80.
[13]
Ariel Farkash, J. T. Timm, and Zeev Waks. 2013. A model-driven approach to clinical practice guidelines representation and evaluation using standards. Studies in Health Technology and Informatics 192 (2013), 200--204.
[14]
N. Galie, N. M. Hoeper, A. Torbicki, J. L Cachiery, J. A. Barbera, M. Beghetti, et al.; ESC Committee for Practice Guidelines (CPG). 2009. Guidelines for the diagnosis and treatment of pulmonary hypertension: The Task Force for the Diagnosis and Treatment of Pulmonary Hypertension of the European Society of Cardiology (ESC) and the European Respiratory Society (ERS), endorsed by the International Society of Heart and Lung Transplantation (ISHLT). European Heart Journal 30, 20 (2009), 2493--2537.
[15]
Kerstin Gerke, Jorge Cardoso, and Alexander Claus. 2009. Measuring the compliance of processes with reference models. In Proceedings of OTM Confederated International Conferences “On the Move to Meaningful Internet Systems.” Springer, 76--93.
[16]
Esther Goldbraich, Zeev Waks, Ariel Farkash, Marco Monti, Michele Torresani, Rossella Bertulli, Paolo Giovanni Casali, and Boaz Carmeli. 2015. Understanding deviations from clinical practice guidelines in adult soft tissue sarcoma. Studies in Health Technology and Informatics 216 (2014), 280--284.
[17]
National Comprehensive Cancer Network. 2015. NCCN Guidelines. Retrieved March 28, 2020 from https://www.nccn.org/professionals/physician_gls/default.aspx.
[18]
Johns Hopkins. 2015. Johns Hopkins Medicine. Retrieved March 28, 2020 from https://www.hopkinsmedicine.org/healthlibrary/.
[19]
Alistair E. W. Johnson, Tom J. Pollard, Lu Shen, Li-Wei H. Lehman, Mengling Feng, Mohammad Ghassemi, Benjamin Moody, et al. 2016. MIMIC-III, a freely accessible critical care database. Scientific Data 3 (2016), 160035.
[20]
James M. Keller, Michael R. Gray, and James A. Givens. 1985. A fuzzy k-nearest neighbor algorithm. IEEE Transactions on Systems, Man, and Cybernetics 4 (1985), 580--585.
[21]
Vinod Khosla. 2014. “20 Percent Doctor Included” 8 Dr. Algorithm: Speculations and Musings of a Technology Optimist. In March 28, 2020 from https://www.khoslaventures.com/20-percent-doctor-included-speculations-and-musings-of-a-technology-optimist.
[22]
Adam Kilgarriff and Christiane Fellbaum. 2000. WordNet: An electronic lexical database. Language 76, 3 (Sept. 2000), 706.
[23]
Amine Labriji, Salma Charkaoui, Issam Abdelbaki, Abdelouhaed Namir, and El Houssine Labriji. 2017. Similarity measure of graphs. International Journal of Recent Contributions from Engineering, Science 8 IT 5, 2 (2017), 42--56.
[24]
Ann Lehman, Norm O’Rourke, Larry Hatcher, and Edward Stepanski. 2005. JMP for Basic Univariate and Multivariate Statistics: A Step-by-Step Guide. SAS Institute Inc., Cary, NC.
[25]
Vladimir I. Levenshtein. 1966. Binary codes capable of correcting deletions, insertions, and reversals. Soviet Physics Doklady 10 (1966), 707--710.
[26]
Axel Martens. 2005. Consistency between executable and abstract processes. In Proceedings of the 2005 IEEE International Conference on e-Technology, e-Commerce, and e-Service (EEE’05). IEEE, Los Alamitos, CA, 60--67.
[27]
Mayo Clinic. 2015. Diseases and Conditions. Retrieved March 28, 2020 from http://www.mayoclinic.org/diseases-conditions.
[28]
John J. V. McMurray, Stamatis Adamopoulos, Stefan D. Anker, Angelo Auricchio, Michael Böhm, Kenneth Dickstein, Volkmar Falk, et al. 2012. ESC guidelines for the diagnosis and treatment of acute and chronic heart failure 2012. European Journal of Heart Failure 14, 8 (2012), 803--869.
[29]
Mirjam Minor, Alexander Tartakovski, and Ralph Bergmann. 2007. Representation and structure-based similarity assessment for agile workflows. In Proceedings of the International Conference on Case-Based Reasoning (ICCBR’07), Vol. 7. 224--238.
[30]
Hans Moen, Juho Heimonen, Laura-Maria Murtola, Antti Airola, Tapio Pahikkala, Virpi Terävä, Riitta Danielsson-Ojala, Tapio Salakoski, and Sanna Salanterä. 2014. On evaluation of automatically generated clinical discharge summaries. In Proceedings of the 2nd European Workshop on Practical Aspects of Health Informatics (PAHI’14). 101--114.
[31]
Jennifer S. Myers, C. Komal Jaipaul, Jennifer R. Kogan, Susan Krekun, Lisa M. Bellini, and Judy A. Shea. 2006. Are discharge summaries teachable? The effects of a discharge summary curriculum on the quality of discharge summaries in an internal medicine residency program. Academic Medicine 81, 10 (2006), S5--S8.
[32]
Lucila Ohno-Machado, John H. Gennari, Shawn N. Murphy, Nilesh L. Jain, Samson W. Tu, Diane E. Oliver, Edward Pattison-Gordon, Robert A. Greenes, Edward H. Shortliffe, and G. Octo Barnett. 1998. The guideline interchange format: A model for representing guidelines. Journal of the American Medical Informatics Association 5, 4 (1998), 357--372.
[33]
S. Pathare, A. Brazinova, and I. Levav. 2018. Care gap: A comprehensive measure to quantify unmet needs in mental health. Epidemiology and Psychiatric Sciences 27, 5 (2018), 1--5.
[34]
Debprakash Patnaik, Patrick Butler, Naren Ramakrishnan, Laxmi Parida, Benjamin J. Keller, and David A. Hanauer. 2011. Experiences with mining temporal event sequences from electronic medical records: Initial successes and some challenges. In Proceedings of the 17th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. ACM, New York, NY, 360--368.
[35]
J. D. Patrick, D. H. M. Nguyen, Y. Wang, and M. Li. 2010. I2b2 challenges in clinical natural language processing 2010. In Proceedings of the 2010 i2b2/VA Workshop on Challenges in Natural Language Processing for Clinical Data.
[36]
Fabian Pedregosa, Gaël Varoquaux, Alexandre Gramfort, Vincent Michel, Bertrand Thirion, Olivier Grisel, Mathieu Blondel, et al. 2011. Scikit-learn: Machine learning in Python. Journal of Machine Learning Research 12 (Oct. 2011), 2825--2830.
[37]
Ian N. Purves. 1998. PRODIGY: Implementing clinical guidance using computers. British Journal of General Practice 48, 434 (1998), 1552.
[38]
Silvana Quaglini, Mario Stefanelli, Giordano Lanzola, Vincenzo Caporusso, and Silvia Panzarasa. 2001. Flexible guideline-based patient careflow systems. Artificial Intelligence in Medicine 22, 1 (2001), 65--80.
[39]
J. Ross Quinlan. 1986. Induction of decision trees. Machine Learning 1, 1 (1986), 81--106.
[40]
Bryan Rink, Sanda Harabagiu, and Kirk Roberts. 2011. Automatic extraction of relations between medical concepts in clinical texts. Journal of the American Medical Informatics Association 18, 5 (2011), 594--600.
[41]
Kirk Roberts, Bryan Rink, and Sanda Harabagiu. 2010. Extraction of medical concepts, assertions, and relations from discharge summaries for the fourth i2b2/VA shared task. In Proceedings of the 2010 i2b2/VA Workshop on Challenges in Natural Language Processing for Clinical Data.
[42]
Natasha M. Rueth, Heather Y. Lin, Isabelle Bedrosian, Simona F. Shaitelman, Naoto T. Ueno, Yu Shen, and Gildy Babiera. 2014. Underuse of trimodality treatment affects survival for patients with inflammatory breast cancer: An analysis of treatment and survival trends from the National Cancer Database. Journal of Clinical Oncology 32, 19 (2014), 2018.
[43]
Sunil Kumar Sahu, Ashish Anand, Krishnadev Oruganty, and Mahanandeeshwar Gattu. 2016. Relation extraction from clinical texts using domain invariant convolutional neural network. arXiv:1606.09370.
[44]
Mihir Shekhar, Veera Ragahvendra Chikka, Lini Thomas, Sunil Mandhan, and Kamalakar Karlapalem. 2015. Identifying medical terms related to specific diseases. In Proceedings of the ICDM Workshop on Biological Data Mining and Its Applications in Healthcare (BioDM’15).
[45]
William F. Styler IV, Steven Bethard, Sean Finan, Martha Palmer, Sameer Pradhan, Piet C. de Groen, Brad Erickson, et al. 2014. Temporal annotation in the clinical domain. Transactions of the Association for Computational Linguistics 2 (2014), 143--154.
[46]
Kjetil Sunde, Morten Pytte, Dag Jacobsen, Arild Mangschau, Lars Petter Jensen, Christian Smedsrud, Tomas Draegni, and Petter Andreas Steen. 2007. Implementation of a standardised treatment protocol for post resuscitation care after out-of-hospital cardiac arrest. Resuscitation 73, 1 (2007), 29--39.
[47]
David R. Sutton and John Fox. 2003. The syntax and semantics of the PRO forma guideline modeling language. Journal of the American Medical Informatics Association 10, 5 (2003), 433--443.
[48]
Kumutha Swampillai and Mark Stevenson. 2011. Extracting relations within and across sentences. In Proceedings of the International Conference on Recent Advances in Natural Language Processing (RANLP’11). 25--32.
[49]
Tom Thaler, Philip Hake, Peter Fettke, and Peter Loos. 2014. Evaluating the evaluation of process matching techniques. In Proceedings of Multikonferenz Wirtschaftsinformatik. 1600--1612.
[50]
Özlem Uzuner, Brett R. South, Shuying Shen, and Scott L. DuVall. 2011. 2010 i2b2/VA challenge on concepts, assertions, and relations in clinical text. Journal of the American Medical Informatics Association 18, 5 (2011), 552--556.
[51]
Karin M. Verspoor, Go Eun Heo, Keun Young Kang, and Min Song. 2016. Establishing a baseline for literature mining human genetic variants and their relationships to disease cohorts. BMC Medical Informatics and Decision Making 16, 1 (2016), 68.
[52]
Zeev Waks, Esther Goldbraich, Ariel Farkash, Michele Torresani, Rossella Bertulli, Nicola Restifo, Paolo Locatelli, Paolo Casali, and Boaz Carmeli. 2013. Analyzing the “careGap”: Assessing gaps in adherence to clinical guidelines in adult soft tissue sarcoma. Students in Health Technology and Informatics 186 (2013), 46--50.
[53]
Fei Wang, Noah Lee, Jianying Hu, Jimeng Sun, and Shahram Ebadollahi. 2012. Towards heterogeneous temporal clinical event pattern discovery: A convolutional approach. In Proceedings of the 18th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. ACM, New York, NY, 453--461.
[54]
Fei Wang, Namyoon Lee, Jianying Hu, Jimeng Sun, Shahram Ebadollahi, and Andrew F. Laine. 2013. A framework for mining signatures from event sequences and its applications in healthcare data. IEEE Transactions on Pattern Analysis and Machine Intelligence 35, 2 (2013), 272--285.
[55]
WebMD. 2015. Health A-Z. Retrieved March 28, 2020 from http://www.webmd.com/a-to-z-guides/common-topics/default.htm.
[56]
WebMD. 2010. Diagnosing Brain Cancer. Retrieved March 28, 2020 from https://www.webmd.com/cancer/brain-cancer/brain-cancer-diagnosis.
[57]
Illhoi Yoo and Min Song. 2008. Biomedical ontologies and text mining for biomedicine and healthcare: A survey. Journal of Computing Science and Engineering 2, 2 (Jun 2008), 109--136.

Index Terms

  1. Conformance Checking Methodology Across Discharge Summaries and Standard Treatment Guidelines

        Recommendations

        Comments

        Information & Contributors

        Information

        Published In

        cover image ACM Transactions on Computing for Healthcare
        ACM Transactions on Computing for Healthcare  Volume 1, Issue 3
        July 2020
        152 pages
        EISSN:2637-8051
        DOI:10.1145/3403604
        Issue’s Table of Contents
        Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

        Publisher

        Association for Computing Machinery

        New York, NY, United States

        Publication History

        Published: 30 May 2020
        Online AM: 07 May 2020
        Accepted: 01 December 2019
        Revised: 01 November 2019
        Received: 01 December 2018
        Published in HEALTH Volume 1, Issue 3

        Permissions

        Request permissions for this article.

        Check for updates

        Author Tags

        1. Treatment plan
        2. conformance measure
        3. discharge summaries
        4. workflow instance extraction
        5. workflow representation

        Qualifiers

        • Research-article
        • Research
        • Refereed

        Funding Sources

        • Hitachi India Pvt. Ltd
        • R8D Centre

        Contributors

        Other Metrics

        Bibliometrics & Citations

        Bibliometrics

        Article Metrics

        • 0
          Total Citations
        • 248
          Total Downloads
        • Downloads (Last 12 months)15
        • Downloads (Last 6 weeks)2
        Reflects downloads up to 20 Feb 2025

        Other Metrics

        Citations

        View Options

        Login options

        Full Access

        View options

        PDF

        View or Download as a PDF file.

        PDF

        eReader

        View online with eReader.

        eReader

        HTML Format

        View this article in HTML Format.

        HTML Format

        Figures

        Tables

        Media

        Share

        Share

        Share this Publication link

        Share on social media