Abstract
Health care organizations have installed electronic systems to increase efficiency in health care. Empirically assessing the cost-effectiveness of technologies to the health care system is a challenging and complex task. This study examined cost-effectiveness of additional clinical information supplied via an EHR system by simulating a case of abdominal aortic aneurysm devised and acted professionally by the Israel Center of Medical Simulation. We conducted a simulation-based study on physicians who were asked to treat a simulated patient for the prevalent medical scenario of hip and leg pain that actually corresponded to an abdominal aortic aneurysm. Half of the participating physicians from the Department of Emergency Medicine at Tel-Hashomer Hospital – Israel’s largest - had access to an EHR system that integrates medical data from multiple health providers (community and hospitals) in addition to the local health record, and half did not. To model medical decision making, the results of the simulation were combined with a Markov Model within a decision tree. Cost-effectiveness was analyzed by comparing the effects of the admission/discharge decision in units of quality adjusted life years (QALYs) to the estimated costs. The results showed that using EHR in the ED increases the QALY of the patient and improves medical decision-making. The expenditure per patient for one QALY unit as a result of using the EHR was $1229, which is very cost-effective according to many accepted threshold values (less than all these values). Thus, using the EHR contributes to making a cost-effective decision in this specific but prevalent case.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Cebul, R. D., Love, T. E., Jain, A. K., and Hebert, C. J., Electronic health records and quality of diabetes care. N. Engl. J. Med. 365(9):825–833, 2011.
Linder, J. A., Schnipper, J. L., and Middleton, B., Method of electronic health record documentation and quality of primary care. J. Am. Med. Inform. Assoc. 19(6):1019–1024, 2012.
Buntin, M. B., Burke, M. F., Hoaglin, M. C., and Blumenthal, D., The benefits of health information technology: A review of the recent literature shows predominantly positive results. Health Aff. 30(3):464–471, 2011.
Jarvis, B., Johnson, T., Butler, P., O’Shaughnessy, K., Fullam, F., Tran, L., and Gupta, R., Assessing the impact of electronic health records as an enabler of hospital quality and patient satisfaction. Acad Med: J Assoc Am Med Coll 88(10):1471–1477, 2013.
Bates, D. W., and Gawande, A. A., Improving safety with information technology. N. Engl. J. Med. 348(25):2526–2534, 2003.
Boonstra, A., and Broekhuis, M., Barriers to the acceptance of electronic medical records by physicians from systematic review to taxonomy and interventions. BMC Health Serv. Res. 10(231):1–17, 2010.
Lluch, M., Healthcare professionals’ organisational barriers to health information technologies—a literature review. Int. J. Med. Inform. 80(12):849–862, 2011.
DesRoches, C. M., Campbell, E. G., Vogeli, C., Zheng, J., Rao, S. R., Shields, A. E., Donelan, K., Rosenbaum, S., Bristol, S. J., and Jha, A. K., Electronic health records’ limited successes suggest more targeted uses. Health Aff. 29(4):639–646, 2010.
Lee, J., Kuo, Y. F., and Goodwin, J. S., The effect of electronic medical record adoption on outcomes in US hospitals. BMC Health Serv. Res. 13(1):39, 2013.
Kannampallil, T. G., Schauer, G. F., Cohen, T., and Patel, V. L., Considering complexity in healthcare systems. J. Biomed. Inform. 44(6):943–947, 2011.
Zeng, Z., Ma, X., Hu, Y., Li, J., and Bryant, D., A simulation study to improve quality of care in the emergency department of a community hospital. J. Emerg. Nurs. 38(4):322–328, 2012.
Fordyce, J., Blank, F. S. J., Pekow, P., Smithline, H. A., Ritter, G., Gehlbach, S., Benjamin, E., and Henneman, P. L., Errors in a busy emergency department. Ann. Emerg. Med. 42(3):324–333, 2003.
Naik, A. D., and Singh, H., Electronic health records to coordinate decision making for complex patients: What can we learn from wiki? Med. Decis. Mak. 30(6):722–731, 2010.
Schiff, G. D., and Bates, D. W., Can electronic clinical documentation help prevent diagnostic errors? N. Engl. J. Med. 362(12):1066–1069, 2010.
Blumenthal, D., Stimulating the adoption of health information technology. N. Engl. J. Med. 360(15):1477–1479, 2009.
Jing, X., Kay, S., Marley, T., Hardiker, N. R., and Cimino, J. J., Incorporating personalized gene sequence variants, molecular genetics knowledge, and health knowledge into an EHR prototype based on the Continuity of Care Record standard. J. Biomed. Inform. 45(1):82–92, 2012.
Mane, K. K., Bizon, C., Schmitt, C., Owen, P., Burchett, B., Pietrobon, R., and Gersing, K., VisualDecisionLinc: A visual analytics approach for comparative effectiveness-based clinical decision support in psychiatry. J. Biomed. Inform. 45(1):101–106, 2012.
Ben-Assuli, O., Shabtai, I., Leshno, M., and Hill, S., EHR in emergency rooms: Exploring the effect of key information components on main complaints. J. Med. Syst. 38(4):1–8, 2014.
Ben-Assuli, O., Leshno, M., and Shabtai, I., Using electronic medical record systems for admission decisions in emergency departments: Examining the crowdedness effect. J. Med. Syst. 36(6):3795–3803, 2012.
Raval, M. V., Rust, L., Thakkar, R. K., Kurtovic, K. J., Nwomeh, B. C., Besner, G. E., and Kenney, B. D., Development and implementation of an electronic health record generated surgical handoff and rounding tool. J. Med. Syst. 39(2):1–8, 2015.
Basu, A., and Meltzer, D., Value of information on preference heterogeneity and individualized care. Med. Decis. Mak. 27(2):112–127, 2007.
Ben-Assuli, O., and Leshno, M., Using electronic medical records in admission decisions: A cost effectiveness analysis. Decis. Sci. 44(3):463–481, 2013.
Takian, A., Sheikh, A., and Barber, N., We are bitter, but we are better off: Case study of the implementation of an electronic health record system into a mental health hospital in England. BMC Health Serv. Res. 12:484, 2012.
Noblin, A., Cortelyou-Ward, K., Cantiello, J., Breyer, T., Oliveira, L., Dangiolo, M., Cannarozzi, M., Yeung, T., and Berman, S., EHR implementation in a new clinic: A case study of clinician perceptions. J. Med. Syst. 37(4):1–6, 2012.
Inverso, G., Flath-Sporn, S. J., Monoxelos, L., Labow, B. I., Padwa, B. L., and Resnick, C. M., What is the cost of meaningful use? J. Oral Maxillofac. Surg. 74(2):227–229, 2016.
Ovretveit, J., Scott, T., Rundall, T. G., Shortell, S. M., and Brommels, M., Improving quality through effective implementation of information technology in healthcare. Int. J. Qual. Health Care 19(5):259–266, 2007.
Adler-Milstein, J., Green, C. E., and Bates, D. W., A survey analysis suggests that electronic health records will yield revenue gains for some practices and losses for many. Health Aff. 32(3):562–570, 2013.
Bar-Dayan, Y., Saed, H., Boaz, M., Misch, Y., Shahar, T., Husiascky, I., and Blumenfeld, O., Using electronic health records to save money. J. Am. Med. Inform. Assoc. 20(e1):e17–e20, 2013.
Bassi, J., and Lau, F., Measuring value for money: A scoping review on economic evaluation of health information systems. J. Am. Med. Inform. Assoc. 20(4):792–801, 2013.
Hripcsak, G., Sengupta, S., Wilcox, A., and Green, R., Emergency department access to a longitudinal medical record. J. Am. Med. Inform. Assoc. 14(2):235–238, 2007.
Hersh, W. R., and Hickam, D. H., How well do physicians use electronic information retrieval systems? JAMA J Am Med Assoc 280(15):1347–1352, 1998.
Laerum, H., Karlsen, T. H., and Faxvaag, A., Effects of scanning and eliminating paper-based medical records on hospital physicians’ clinical work practice. J. Am. Med. Inform. Assoc. 10(6):588–595, 2003.
Walter, Z., and Lopez, M. S., Physician acceptance of information technologies: Role of perceived threat to professional autonomy. Decis. Support. Syst. 46(1):206–215, 2008.
Ziv, A., Erez, D., Munz, Y., Vardi, A., Barsuk, D., Levine, I., Benita, S., Rubin, O., and Berkenstadt, H., The Israel Center for Medical Simulation: A paradigm for cultural change in medical education. Acad. Med. 81(12):1091–1097, 2006.
Chun, K. C., Teng, K. Y., Chavez, L. A., Van Spyk, E. N., Samadzadeh, K. M., Carson, J. G., and Lee, E. S., Risk factors associated with the diagnosis of abdominal aortic aneurysm in patients screened at a regional Veterans Affairs health care system. Ann. Vasc. Surg. 28(1):87–92, 2014.
Hye, R. J., Smith, A. E., Wong, G. H., Vansomphone, S. S., Scott, R. D., and Kanter, M. H., Leveraging the electronic medical record to implement an abdominal aortic aneurysm screening program. J. Vasc. Surg. 59(6):1535–1543, 2014.
Pliskin, J. S., Towards better decision making in growth hormone therapy. Horm. Res. 51:30–35, 1999.
Sonnenberg, F. A., and Beck, J. R., Markov models in medical decision making: A practical guide. Med. Decis. Mak. 13(4):322–338, 1993.
Blackhouse, G., Hopkins, R., Bowen, J. M., De Rose, G., Novick, T., Tarride, J. E., O’Reilly, D., Xie, F., and Goeree, R., Cost-effectiveness model comparing endovascular repair to open surgical repair of abdominal aortic aneurysms in Canada. Value Health 12(2):245–252, 2009.
Lederle, F. A., Freischlag, J. A., Kyriakides, T. C., Matsumura, J. S., Padberg, F. T., Kohler, T. R., Kougias, P., Jean-Claude, J. M., Cikrit, D. F., Swanson, K. M., OVER Veterans Affairs Cooperative Study Group, Long-term comparison of endovascular and open repair of abdominal aortic aneurysm. N. Engl. J. Med. 367(21):1988–1997, 2012.
Powell, J. T., and Greenhalgh, R., Small abdominal aortic aneurysms. N. Engl. J. Med. 348(19):1895–1901, 2003.
Weibull, W., A statistical distribution function of wide applicability. J Appl Mech ASME 18(3):293–297, 1951.
Achcar, J. A., Piratelli, C. L., and de Souza, R. M., Modeling quality control data using Weibull distributions in the presence of a change point. Int. J. Adv. Manuf. Technol. 1 (11), 2013.
Williams, A., Economics, QALYs and medical ethics—a health economist’s perspective. Health Care Anal. 3(3):221–226, 1995.
Golan, Y., Wolf, M. P., Pauker, S. G., Wong, J. B., and Hadley, S., Empirical anti-Candida therapy among selected patients in the intensive care unit: A cost-effectiveness analysis. Ann. Intern. Med. 143(12):857–869, 2005.
Guyatt, G., Baumann, M., Pauker, S., Halperin, J., Maurer, J., Owens, D. K., Tosteson, A. N. A., Carlin, B., Gutterman, D., and Prins, M., Addressing resource allocation issues in recommendations from clinical practice guideline panels suggestions from an american college of chest physicians task force. Am Coll Chest Phys 129(1):182–187, 2006.
Leshno, M., Halpern, Z., and Arber, N., Cost-effectiveness of colorectal cancer screening in the average risk population. Health Care Manag Sci 6(3):165–174, 2003.
Shamir, R., Hernell, O., and Leshno, M., Cost-effectiveness analysis of screening for celiac disease in the adult population. Med. Decis. Mak. 26(3):282–293, 2006.
Kim, S. H., and Netessine, S., Collaborative cost reduction and component procurement under information asymmetry. Manag. Sci. 59(1):189–206, 2013.
Acknowledgments
The research was fully funded by a grant from the Israel National Institute for Health Policy Research. This article is based on the corresponding author’s presentations at the International Conference on Information Systems (ICIS 2014 in Auckland, New Zealand), and at the Israel National Institute for Health Policy Research (Israel).
Author information
Authors and Affiliations
Corresponding author
Additional information
This article is part of the Topical Collection on Systems-Level Quality Improvement
Appendices
Appendices
Appendix 1 OFEK system (EHR system)
The system used in this study is known as “OFEK” and was developed by dbMotion. The system allows health care organizations to share medical information and create a Virtual Patient Record. The system connects health care providers, and thus ensures the system’s interoperability within the organization and with other organizations. It collects medical information from available systems (distributed health care providers, external labs, pharmacies, hospitals, medical institutes and community clinics). Then, the information is integrated into a Virtual Patient Object (VPO) – a representation of the patient’s information in the system. After the VPO is formed, OFEK can analyze the information existing in the system to search for additional information by identifying the patient. The information sources include prior visits and hospitalizations, prior diagnoses, medication lists, allergies, previous lab results, etc.
Appendix 2
Appendix 3
Appendix 4 Sensitivity analyses on the results
We used general sensitivity analysis and we also applied a Monte Carlo simulation. The analysis below compares admission and discharge decisions with or without access to the OFEK.
First-order sensitivity analysis tests the elasticity of the model’s results in response to a change in a single variable while all remaining variables are held constant. We performed general sensitivity analyses after identifying the most influential variables using a Tornado simulation. The Tornado analysis identifies the most influential variables, and shows how possible changes in the variables affect decision and diagnosis (with and without OFEK). The diagram displays the horizontal bars in descending order of influence (the bar for the variable with the greatest impact on the result appears at the top).
Based on our basic decision tree model, we constructed a second model, Monte Carlo Simulation, comprising variables that represent different types of distributions rather than specific values.
The figures and tables below show the results of a diverse set of sensitivity analyses. We first present the Tornado sensitivity analysis results, followed by the results of a set of one-way and two-ways sensitivity analyses on the variables. Then, we present the results of the Monte Carlo sensitivity analysis, including the results of the cost evaluation sensitivity analysis.
All sensitivity analyses were run for each OFEK access mode (with or without OFEK).
The right-hand side of Fig. 3 lists the nine variables that were analyzed for sensitivity, in descending order of impact level: p_DIE_AAA_ADMIT, p_admit_with_ofek, Time Horizon, gamma, p_diagnosed_AAA, lambda, LE_Delay, p_DIE_AAA_DISCHARGE and p_admit_without_ofek. In general, all the variables had a relatively limited impact on the access mode, since the utility values ranged from 26 to 24.9 years. As seen in Fig. 3, using p_DIE_AAA_ADMIT variable had the largest potential effect on the expected value of the model.
Tornado sensitivity analysis
Figure 4 shows that in all p_diagnosed_AAA values chosen for the analysis, using OFEK dominated.
Sensitivity analysis for the probability of diagnosing AAA
Figure 5 shows that for all the values of p_admit_with_ofek and p_admit_without_ofek, accessing OFEK was dominant.
Sensitivity analysis for the probabilities of admission with and without OFEK
Figure 6 shows that for all values of gamma and lambda (Weibull equation) using OFEK was dominant.
Sensitivity analysis for gamma and lambda
Figure 7 shows that for all values of the patients’ Time Horizon, in all examined values of the delay in life expectancy, OFEK use was dominant.
Sensitivity analysis for time horizon and LE delay
Figure 8 shows that for all values of p_die_AAA_ADMIT and p_DIE_AAA_DISCHARGE, using OFEK was dominant.
Sensitivity analysis for probabilities of dying from AAA in admit and discharge decisions
Figure 9 shows that for all values of cAdmitAAA and cDischargeAAA, not accessing OFEK was dominant (less costly).
Sensitivity analysis for the cost of admit or discharge after AAA
Figure 10 shows that for all values of cAdmitAAADie, not accessing OFEK was dominant (less costly). It also shows that for each cAdmitAAADie values, the expected values of with OFEK were much higher (between $29,000 to $30,000) than the expected values for each cAdmitAAADie values without OFEK (between $14,000 to $15,000).
Sensitivity analysis of costs after an admit decision following an AAA diagnosis when the patient dies
Monte Carlo sensitivity analysis on case 1 cost values
We used 150,000 trials to run a sensitivity analysis on the distributions of the variables, the cost evaluations (with and without EHR), and their sensitivity ranges.
Sensitivity analysis and Monte-Carlo simulation
Statistics | LE-Utility with OFEK (in years) | LE-Utility without OFEK (in years) | Costs with OFEK ($) | Costs without OFEK ($) |
|---|---|---|---|---|
Mean | 25.47 | 13.1 | 29492.49 | 14249.63 |
Std Deviation | 11.38 | 12.68 | 2662.66 | 9076.59 |
Minimum | 0 | 0 | 0 | 0 |
10 % | 8 | 0 | 30,000 | 11,000 |
Median | 27 | 8 | 30,000 | 11,000 |
90 % | 40 | 30 | 30,000 | 30,000 |
Maximum | 40 | 40 | 30,000 | 30,000 |
Variance | 129.58 | 160.85 | 7089783.6 | 82384433.2 |
Variance/Size | 0.001 | 0.001 | 47.27 | 549.23 |
SQRT[Variance/Size] | 0.020 | 0.021 | 6.88 | 23.44 |
The mean utility with OFEK and utility without OFEK costs were $29,492.49 and $14,249.63, respectively (mean ∆Costs = $15,242.86). The mean utility with OFEK and utility without OFEK QALY units were 25.47 and 13.1, respectively (mean ∆QALY = $12.37). Thus, the ICER was $1232.24.
Rights and permissions
About this article
Cite this article
Ben-Assuli, O., Ziv, A., Sagi, D. et al. Cost-Effectiveness Evaluation of EHR: Simulation of an Abdominal Aortic Aneurysm in the Emergency Department. J Med Syst 40, 141 (2016). https://doi.org/10.1007/s10916-016-0502-9
Received:
Accepted:
Published:
DOI: https://doi.org/10.1007/s10916-016-0502-9