Skip to main content

Advertisement

SpringerLink
Log in

Evaluation Study for an ISO 13606 Archetype Based Medical Data Visualization Method

  • Patient Facing Systems
  • Published:
Journal of Medical Systems Aims and scope Submit manuscript

Abstract

The objective of this evaluation study is to assess a method for standard based medical data visualization. The method allows flexible and customizable visualization for ISO 13606 archetype based medical data. The chosen evaluation concept is based the Guideline for Good Evaluation Practice in Health Informatics (GEP-HI). The stages of the study were identified. Each stage got a detailed description. We also identified the participants and their required qualifications and responsibilities. The evaluation location was described in details. The evaluation metrics were defined. The questionnaires for doctors, patients and experts were developed to fulfill the requirements of the evaluation study. The study was performed in Tomsk, Russia. 30 patients and 5 doctors participated in the study. The overall performance of the users reached the expert level by the end of the study. Patients as well as medical staff stated in their comments that the usability of the system was high, and they preferred it to the previously used paper-based and computer based systems. This was also shown by the high level of satisfaction measured within our study. The visualization approach, integrated into the electronic health record, was well accepted in our pilot setting with high usability scores from patients and doctors alike. The results showed the efficiency for both modeling and visualization part of the system

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

References

  1. Tse, J., and You, W., How accurate is the electronic health record? - a pilot study evaluating information accuracy in a primary care setting. Stud Health Technol Inform 168:158–164, 2011.

    CAS  PubMed  Google Scholar 

  2. McGowan, J. J., Cusack, C. M., and Poon, E. G., Formative evaluation: a critical component in EHR implementation. J Am Med Inform Assoc 15:297–301, 2008.

    Article  PubMed Central  PubMed  Google Scholar 

  3. Huang, Z., Bao, Y., Dong, W., Lu, X., and Duan, H., Online treatment compliance checking for clinical pathways. J Med Syst 38:123, 2014.

    Article  PubMed  Google Scholar 

  4. Pelone, F., Kringos, D. S., Romaniello, A., Archibugi, M., Salsiri, C., and Ricciardi, W., Primary care efficiency measurement using data envelopment analysis: a systematic review. J Med Syst 39:156, 2015.

    Article  PubMed  Google Scholar 

  5. Top, M., Yilmaz, A., Karabulut, E., Otieno, O. G., Saylam, M., Bakir, S., and Top, S., Validation of a nurses’ views on electronic medical record systems (EMR) questionnaire in Turkish health system. J Med Syst 39:67, 2015.

    Article  PubMed  Google Scholar 

  6. Buenestado, D., Elorz, J., Perez-Yarza, E. G., Iruetaguena, A., Segundo, U., Barrena, R., and Pikatza, J. M., Evaluating Acceptance and User Experience of a Guideline-based Clinical Decision Support System Execution Platform. J Med Syst 37:9910, 2013.

    Article  PubMed  Google Scholar 

  7. Jin, H. S., Kim, M. H., Lee, S. Y., Jeong, H. Y., Choi, S. J., and Lee, H. W., A survey of user acceptance of electronic patient anesthesia records. Korean J Anesthesiol 62:350–357, 2012.

    Article  PubMed Central  PubMed  Google Scholar 

  8. Simon-Arndt, C. M., Hurtado, S. L., and Patriarca-Troyk, L. A., Acceptance of Web-based personalized feedback: user ratings of an alcohol misuse prevention program targeting U.S. Marines. Health Commun 20:13–22, 2006.

    Article  CAS  PubMed  Google Scholar 

  9. Ertmer, A., and Uckert, F., User acceptance of and satisfaction with a personal electronic health record. Stud Health Technol Inform 116:861–866, 2005.

    CAS  PubMed  Google Scholar 

  10. Zapata, B. C., Fernandez-Aleman, J. L., Idri, A., and Toval, A., Empirical studies on usability of mHealth apps: a systematic literature review. J Med Syst 39:1, 2015.

    Article  PubMed  Google Scholar 

  11. Zhang, L., and Zhu, S., Robust ECC-based authenticated key agreement scheme with privacy protection for Telecare medicine information systems. J Med Syst 39:49, 2015.

    Article  PubMed  Google Scholar 

  12. Lorenzi, N. M., and Riley, R. T., Managing change: an overview. J Am Med Inform Assoc 7:116–124, 2000.

    Article  CAS  PubMed Central  PubMed  Google Scholar 

  13. Sockolow, P. S., Crawford, P. R., and Lehmann, H. P., Health services research evaluation principles. Broadening a general framework for evaluating health information technology. Methods Inf Med 51:122–130, 2012.

    Article  CAS  PubMed  Google Scholar 

  14. Kushniruk, A., Nohr, C., Jensen, S., and Borycki, E. M., From Usability Testing to Clinical Simulations: Bringing Context into the Design and Evaluation of Usable and Safe Health Information Technologies. Contribution of the IMIA Human Factors Engineering for Healthcare Informatics Working Group. Yearb Med Inform 8:78–85, 2013.

    CAS  PubMed  Google Scholar 

  15. Abidi, S. R., Stewart, S., Shepherd, M., and Abidi, S. S., Usability Evaluation of Family Physicians’ Interaction with the Comorbidity Ontological Modeling and ExecuTion System (COMET). Stud Health Technol Inform 192:447–451, 2013.

    PubMed  Google Scholar 

  16. Akin, B. A., Bryson, S. A., Testa, M. F., Blase, K. A., McDonald, T., and Melz, H., Usability testing, initial implementation, and formative evaluation of an evidence-based intervention: Lessons from a demonstration project to reduce long-term foster care. Eval Program Plann 41C:19–30, 2013.

    Article  Google Scholar 

  17. Voncken-Brewster, V., Moser, A., van der Weijden, T., Nagykaldi, Z., de Vries, H., and Tange, H., Usability evaluation of an online, tailored self-management intervention for chronic obstructive pulmonary disease patients incorporating behavior change techniques. JMIR Res Protoc 2:e3, 2013.

    Article  PubMed Central  PubMed  Google Scholar 

  18. Kopanitsa, G., Tsvetkova, Z., and Veseli, H., Analysis of metrics for the usability evaluation of EHR management systems. Stud Health Technol Inform 180:358–362, 2012.

    PubMed  Google Scholar 

  19. Kopanitsa, G., Tsvetkova, Z., and Veseli, H., Analysis of metrics for the usability evaluation of electronic health record systems. Stud Health Technol Inform 174:129–133, 2012.

    PubMed  Google Scholar 

  20. Berk, L., Berk, M., Dodd, S., Kelly, C., Cvetkovski, S., and Jorm, A. F., Evaluation of the acceptability and usefulness of an information website for caregivers of people with bipolar disorder. BMC Med 11:162, 2013.

    Article  PubMed Central  PubMed  Google Scholar 

  21. Drescher, F. S., Chandrika, S., Weir, I. D., Weintraub, J. T., Berman, L., Lee, R., Van Buskirk, P. D., Wang, Y., Adewunmi, A., and Fine, J. M., Effectiveness and acceptability of a computerized decision support system using modified Wells criteria for evaluation of suspected pulmonary embolism. Ann Emerg Med 57:613–621, 2011.

    Article  PubMed  Google Scholar 

  22. Nabovati, E., Vakili-Arki, H., Eslami, S., and Khajouei, R., Usability evaluation of Laboratory and Radiology Information Systems integrated into a hospital information system. J Med Syst 38:35, 2014.

    Article  PubMed  Google Scholar 

  23. Ouhbi, S., Fernandez-Aleman, J. L., Pozo, J. R., Bajta, M. E., Toval, A., and Idri, A., Compliance of blood donation apps with mobile OS usability guidelines. J Med Syst 39:63, 2015.

    Article  PubMed  Google Scholar 

  24. Kopanitsa, G., Standard based multiclient medical data visualization. Stud Health Technol Inform 180:199–203, 2012.

    PubMed  Google Scholar 

  25. Huebner-Bloder, G., Duftschmid, G., Kohler, M., Rinner, C., Saboor, S., and Ammenwerth, E., An EHR prototype using structured ISO/EN 13606 documents to respond to identified clinical information needs of diabetes specialists: a controlled study on feasibility and impact. AMIA Annu Symp Proc 2012:380–389, 2012.

    PubMed Central  PubMed  Google Scholar 

  26. Sun, S., Austin, T., and Kalra, D., A data types profile suitable for use with ISO EN 13606. J Med Syst 36:3621–3635, 2012.

    Article  PubMed  Google Scholar 

  27. Kopanitsa, G., Evaluating study for a multiuser oriented medical data visualization method. Stud Health Technol Inform 197:123, 2014.

    PubMed  Google Scholar 

  28. Kopanitsa, G., Evaluation study for a multi-user oriented medical data visualization method. Stud Health Technol Inform 200:158–160, 2014.

    PubMed  Google Scholar 

  29. Kopanitsa, G., Hildebrand, C., Stausberg, J., and Englmeier, K. H., Visualization of medical data based on EHR standards. Methods Inf Med 52:43–50, 2013.

    Article  CAS  PubMed  Google Scholar 

  30. Kopanitsa, G., and Taranik, M., Applying open source data visualization tools to standard based medical data. Stud Health Technol Inform 200:155–157, 2014.

    PubMed  Google Scholar 

  31. Kopanitsa G, Veseli H, Yampolsky V: Development, implementation and evaluation of an information model for archetype based user responsive medical data visualization. J Biomed Inform 2015.

  32. Nykanen, P., Brender, J., Talmon, J., de Keizer, N., Rigby, M., Beuscart-Zephir, M. C., and Ammenwerth, E., Guideline for good evaluation practice in health informatics (GEP-HI). Int J Med Inform 80:815–827, 2011.

    Article  PubMed  Google Scholar 

  33. Nykanen, P., Brender, J., Ammenwerth, E., Talmon, J., and Rigby, M., Applying GEP-HI for the planning of an evaluation study: a case study walkthrough (workshop). Stud Health Technol Inform 174:134–136, 2012.

    PubMed  Google Scholar 

  34. Veseli, H., Kopanitsa, G., and Demski, H., Standardized EHR interoperability - preliminary results of a German pilot project using the archetype methodology. Stud Health Technol Inform 180:646–650, 2012.

    PubMed  Google Scholar 

  35. McKinlay, A., McVittie, C., Reiter, E., Freer, Y., Sykes, C., and Logie, R., Design issues for socially intelligent user interfaces. A discourse analysis of a data-to-text system for summarizing clinical data. Methods Inf Med 49:379–387, 2010.

    Article  CAS  PubMed  Google Scholar 

  36. Bird, L., Brooks, C., Cheong, Y. C., and Tun, N. N., A logical approach to semantic interoperability in healthcare. Stud Health Technol Inform 168:1–9, 2011.

    PubMed  Google Scholar 

  37. Kopanitsa, G., Arranging ISO 13606 archetypes into a knowledge base. Stud Health Technol Inform 205:33–37, 2014.

    PubMed  Google Scholar 

  38. Kopanitsa, G., and Yampolskiy, V., Approach to extract billing data from medical documentation in Russia - lessons learned. Stud Health Technol Inform 210:349–353, 2015.

    PubMed  Google Scholar 

  39. Kopanitsa, G., and Yampolsky, V., Application of ISO 13606 Archetypes for a HIS-LIS integration. Stud Health Technol Inform 210:1018, 2015.

    Google Scholar 

  40. Shrestha, B. M., The Declaration of Helsinki in relation to medical research: historical and current perspectives. J Nepal Health Res Counc 10:254–257, 2012.

    CAS  PubMed  Google Scholar 

  41. Beyleveld, D., and Townend, D. M., When is personal data rendered anonymous? Interpreting Recital 26 of Directive 95/46/EC. Med Law Int 6:73–86, 2004.

    Article  PubMed  Google Scholar 

  42. Amith, M., Loubser, P. G., Chapman, J., Zoker, K. C., and Rabelo Ferreira, F. E., Optimization of an EHR Mobile Application Using the UFuRT Conceptual Framework. AMIA Annu Symp Proc 2012:209–217, 2012.

    PubMed Central  PubMed  Google Scholar 

  43. Edinger, T., Cohen, A. M., Bedrick, S., Ambert, K., and Hersh, W., Barriers to Retrieving Patient Information from Electronic Health Record Data: Failure Analysis from the TREC Medical Records Track. AMIA Annu Symp Proc 2012:180–188, 2012.

    PubMed Central  PubMed  Google Scholar 

  44. Ferrandez, O., South, B. R., Shen, S., Friedlin, F. J., Samore, M. H., and Meystre, S. M., Evaluating current automatic de-identification methods with Veteran’s health administration clinical documents. BMC Med Res Methodol 12:109, 2012.

    Article  PubMed Central  PubMed  Google Scholar 

  45. Liu J, Luo L, Zhang R, Huang T: Patient satisfaction with electronic medical/health record: a systematic review. Scand J Caring Sci ,2012.

  46. Richardson, J. E., Abramson, E. L., Pfoh, E. R., and Kaushal, R., Bridging informatics and implementation science: evaluating a framework to assess electronic health record implementations in community settings. AMIA Annu Symp Proc 2012:770–778, 2012.

    PubMed Central  PubMed  Google Scholar 

  47. Christmann, A., and Van Aelst, S., Robust estimation of Cronbach’s alpha. J Multivar Anal 97:1660–1674, 2006.

    Article  Google Scholar 

  48. Faxvaag, A., Toussaint, P., and Johansen, T. S., Research management in healthcare informatics - experiences from Norway. Stud Health Technol Inform 169:980–984, 2011.

    PubMed  Google Scholar 

  49. Atkinson, A. C., Donev, A. N., and Tobias, R. D., Optimum experimental designs, with SAS. Oxford University Press, Oxford, 2007.

    Google Scholar 

  50. Abramson, E. L., Patel, V., Malhotra, S., Pfoh, E. R., Nena Osorio, S., Cheriff, A., Cole, C. L., Bunce, A., Ash, J., and Kaushal, R., Physician experiences transitioning between an older versus newer electronic health record for electronic prescribing. Int J Med Inform 81:539–548, 2012.

    Article  PubMed  Google Scholar 

  51. Bar-Dayan, Y., Saed, H., Boaz, M., Misch, Y., Shahar, T., Husiascky, I., and Blumenfeld, O., Using electronic health records to save money. J Am Med Inform Assoc 20:e17–e20, 2013.

    Article  PubMed Central  PubMed  Google Scholar 

  52. Bramble, J. D., Galt, K. A., Siracuse, M. V., Abbott, A. A., Drincic, A., Paschal, K. A., and Fuji, K. T., The relationship between physician practice characteristics and physician adoption of electronic health records. Health Care Manage Rev 35:55–64, 2010.

    Article  PubMed  Google Scholar 

  53. Christensen, T., and Grimsmo, A., Instant availability of patient records, but diminished availability of patient information: a multi-method study of GP’s use of electronic patient records. BMC Med Inform Decis Mak 8:12, 2008.

    Article  PubMed Central  PubMed  Google Scholar 

  54. Hackl, W. O., Hoerbst, A., and Ammenwerth, E., Why the hell do we need electronic health records? EHR acceptance among physicians in private practice in Austria: a qualitative study. Methods Inf Med 50:53–61, 2011.

    Article  CAS  PubMed  Google Scholar 

  55. Klimov, D., Shahar, Y., and Taieb-Maimon, M., Intelligent interactive visual exploration of temporal associations among multiple time-oriented patient records. Methods Inf Med 48:254–262, 2009.

    Article  PubMed  Google Scholar 

  56. Lanham, H. J., Leykum, L. K., and McDaniel, R. R., Jr., Same organization, same electronic health records (EHRs) system, different use: exploring the linkage between practice member communication patterns and EHR use patterns in an ambulatory care setting. J Am Med Inform Assoc 19:382–391, 2012.

    Article  PubMed Central  PubMed  Google Scholar 

  57. Le, T., Reeder, B., Thompson, H., and Demiris, G., Health providers’ perceptions of novel approaches to visualizing integrated health information. Methods Inf Med 52:250–258, 2013.

    Article  CAS  PubMed Central  PubMed  Google Scholar 

  58. McGinn, C. A., Grenier, S., Duplantie, J., Shaw, N., Sicotte, C., Mathieu, L., Leduc, Y., Legare, F., and Gagnon, M. P., Comparison of user groups’ perspectives of barriers and facilitators to implementing electronic health records: a systematic review. BMC Med 9:46, 2011.

    Article  PubMed Central  PubMed  Google Scholar 

  59. Kuijpers, W., Groen, W. G., Aaronson, N. K., and van Harten, W. H., A systematic review of web-based interventions for patient empowerment and physical activity in chronic diseases: relevance for cancer survivors. J Med Internet Res 15:e37, 2013.

    Article  PubMed Central  PubMed  Google Scholar 

  60. White, R. D., Patient empowerment and optimal glycemic control. Curr Med Res Opin 28:979–989, 2012.

    Article  CAS  PubMed  Google Scholar 

  61. Lejbkowicz, I., Caspi, O., and Miller, A., Participatory medicine and patient empowerment towards personalized healthcare in multiple sclerosis. Expert Rev Neurother 12:343–352, 2012.

    Article  PubMed  Google Scholar 

  62. Meyer, C., Muhlfeld, A., Drexhage, C., Floege, J., Goepel, E., Schauerte, P., Kelm, M., and Rassaf, T., Clinical research for patient empowerment--a qualitative approach on the improvement of heart health promotion in chronic illness. Med Sci Monit 14:CR358–CR365, 2008.

    PubMed  Google Scholar 

  63. Kumar, S., Client empowerment in psychiatry and the professional abuse of clients: where do we stand? Int J Psychiatry Med 30:61–70, 2000.

    Article  CAS  PubMed  Google Scholar 

  64. Rinner, C., Janzek-Hawlat, S., Sibinovic, S., and Duftschmid, G., Semantic validation of standard-based electronic health record documents with W3C XML schema. Methods Inf Med 49:271–280, 2010.

    Article  CAS  PubMed  Google Scholar 

  65. K A, HY Y: From openEHR Domain Models to Advanced User Interfaces: A Case Study in Endoscopy. In 2010 Health Informatics New Zealand Conference. Wellington, 2010.

  66. van der Linden, H., Austin, T., and Talmon, J., Generic screen representations for future-proof systems, is it possible? There is more to a GUI than meets the eye. Comput Methods Programs Biomed 95:213–226, 2009.

    Article  PubMed  Google Scholar 

Download references

Acknowledgements

We thank the Bavarian State Ministry of the Environment and Public Health (http://www.stmug.bayern.de) for the financial support of the project.

Conflict of Interest

The authors declare that they have no conflict of interests.

Author information

Authors and Affiliations

  1. Institute Cybernetic Center, Tomsk Polytechnic University, Lenina 30, 634050, Tomsk, Russia

    Georgy Kopanitsa

  2. Tomsk State University of Architecture and Building, Solyanaya 2, Tomsk, Russia

    Georgy Kopanitsa

Authors
  1. Georgy Kopanitsa
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Georgy Kopanitsa.

Additional information

This article is part of the Topical Collection on Patient Facing Systems

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Kopanitsa, G. Evaluation Study for an ISO 13606 Archetype Based Medical Data Visualization Method. J Med Syst 39, 82 (2015). https://doi.org/10.1007/s10916-015-0270-y

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s10916-015-0270-y

Keywords

Search

Navigation