research-article

Documentation quality and time costs: A randomized controlled trial of structured entry versus dictation

Authors:

Trent S. Rosenbloom,

Shawn P. Hardenbrook,

Elliot Fielstein,

Ted SperoffAuthors Info & Claims

Journal of Data and Information Quality (JDIQ), Volume 3, Issue 1

Article No.: 2, Pages 1 - 17

https://doi.org/10.1145/2166788.2166790

Published: 07 May 2012 Publication History

Abstract

The Department of Veterans Affairs (VA) performs over 800,000 disability exams and distributes over $37 billion in disability benefits per year. VA developed and deployed a computer-based disability exam documentation system in order to improve exam report quality and timeliness. We conducted a randomized controlled trial comparing joint disability examinations supported by computerized templates to the examinations documented via dictation, to determine if the system met the intended goals or had unintended consequences. Consenting veterans were randomized to undergo exams documented using computerized templates or via dictation. We compared exam report quality, documentation time costs, encounter length, total time to fulfill an exam request with a finalized exam report, and veteran satisfaction. Computer-based templates resulted in disability exam reports that had higher quality scores (p. 0.042) and were returned to the requesting office faster than exam reports created via dictation (p. 0.02).

Documentation time and veteran satisfaction were similar for both the documentation techniques. Encounter length was significantly longer for the template group. Computer-based templates impacted the VA disability evaluation system by improving report quality scores and production time and lengthening encounter times. Oversight bodies have called for mandated use of computer-based templates nationwide. We believe mandates regarding use of health information technology should be guided by data regarding its positive and negative impacts.

References

[1]

Ahlers, P., Sullivan, R. J., Hammond, W. E., Walter, E. L., and Tolley, H. D. 1976. The cost of auditing outpatient records. South Med. J. 69, 1328--1330.

[2]

Apkon, M. and Singhaviranon, P. 2001. Impact of an electronic information system on physician workflow and data collection in the intensive care unit. Intensive Care Med. 27, 122--130.

[3]

Aronsky, D. and Haug, P. J. 2000. Assessing the quality of clinical data in a computer-based record for calculating the pneumonia severity index. J. Amer. Med. Inform. Assoc. 7, 55--65.

[4]

Ash, J. S., Berg, M., and Coiera, E. 2003. Some unintended consequences of information technology in health care: The nature of patient care informations system related errors. J. Amer. Med. Inform. Assoc. Preprint.

[5]

Barnett, G. O., Winickoff, R., Dorsey, J. L., Morgan, M. M., and Lurie, R. S. 1978. Quality assurance through automated monitoring and concurrent feedback using a computer-based medical information system. Med. Care 16, 962--970.

[6]

Barnett, G. O., Winickoff, R. N., Morgan, M. M., and Zielstorff, R. D. 1983. A computer-based monitoring system for follow-up of elevated blood pressure. Med Care 21, 400--409.

[7]

Bell, D. S. and Greenes, R. A. 1994. Evaluation of UltraSTAR: performance of a collaborative structured data entry system. In Proceedings of the Annual Symposium on Computer Applications Medical Care. 216--222.

[8]

Blake, R. and Mangiameli, P. 2009. The effects and interactions of data quality and problem complexity on data mining. In Proceedings of the 13th International Conference on Information Quality (ICIQ), 160--165.

[9]

Brown, S. H., Hardenbrook, S., Herrick, L., St. Onge, J., Bailey, K., and Elkin, P. L. 2001. Usability evaluation of the progress note construction set. In Proceedings of the AMIA Symposium. 76--80.

[10]

Brown, S. H., Lincoln, M. J., Groen, P. J., and Kolodner, R. M. 2003. VistA--U.S. Department of Veterans Affairs national-scale HIS. Int. J. Med. Inf. 69, 135--156.

[11]

Brown, S. H., Speroff, T., Fielstein, E. M., Bauer, B. A., Wahner-Roedler, D. L., Greevy, R., and Elkin, P. L. 2006. eQuality: Electronic quality assessment from narrative clinical reports. Mayo Clin. Proc. 81, 1472--1481.

[12]

Cimino, J. J., Johnson, S. B., Aguirre, A., Roderer, N., and Clayton, P. D. 1992. The MEDLINE button. In Proceedings of the Annual Symposium on Computer Applications Medical Care. 81--85.

[13]

Cimino, J. J., Patel, V. L., and Kushniruk, A. W. 2001. Studying the human-computer-terminology interface. J Amer. Med. Inform. Assoc. 8, 163--173.

[14]

Dawdy, M. R., Munter, D. W., and Gilmore, R. A. 1997. Correlation of patient entry rates and physician documentation errors in dictated and handwritten emergency treatment records. Amer. J. Emerg. Med. 15, 115--117.

[15]

Dollery, C. T., Beilin, L. J., Bulpitt, C. J., Coles, E. C., Johnson, B. F., Munro-Faure, A. D., and Turner, S. C. 1977. Initial care of hypertensive patients. Influence of different types of clinical records. British Heart J. 39, 181--185.

[16]

Duggan, A. K., Starfield, B., and Deangelis, C. 1990. Structured encounter form: The impact on provider performance and recording of well-child care. Pediatrics 85, 104--113.

[17]

Essin, D. J., Dishakjian, R., Deciutiis, V. L., Essin, C. D., and Steen, S. N. 1998. Development and assessment of a computer-based preanesthetic patient evaluation system for obstetrical anesthesia. J. Clin. Monit. Comput. 14, 95--100.

[18]

Evans, R. S., Pestotnik, S. L., Classen, D. C., Clemmer, T. P., Weaver, L. K., Orme, J. F., Jr., Lloyd, J. F., and Burke, J. P. 1998. A computer-assisted management program for antibiotics and other antiinfective agents. New. Eng. J. Med. 338, 232--238.

[19]

Fatzinger, P., Kammer, A., and Garrett, M. 1992. Development and use of preprinted forms and adhesive labels in medical record charting. J. Amer. Diet Assoc. 92, 982--985.

[20]

Fielstein, E. M., Brown, S. H., McBrine, C. S., Clark, T. K., Hardenbrook, S. P., and Speroff, T. 2006. The effect of standardized computer-guided templates on quality of VA disability exams. In Proceedings of the American Medical Information Association Fall Symposium. 249--253.

[21]

Geissbuhler, A. and Miller, R. A. 1996. A new approach to the implementation of direct care-provider order entry. In Proceedings of the AMIA Annual Fall Symposium. 689--693.

[22]

Gouveia-Oliveira, A., Raposo, V. D., Salgado, N. C., Almeida, I., Nobre-Leitao, C., and De Melo, F. G. 1991. Longitudinal comparative study on the influence of computers on reporting of clinical data. Endoscopy 23, 334--337.

[23]

Gray, J. E., Safran, C., Davis, R. B., Pompilio-Weitzner, G., Stewart, J. E., Zaccagnini, L., and Pursley, D. 2000. Baby CareLink: Using the internet and telemedicine to improve care for high-risk infants. Pediatrics 106, 1318--1324.

[24]

Hammond, K. W. and Munnecke, T. H. 1984. A computerized psychiatric treatment planning system. Hosp. Comm. Psych. 35, 160--163.

[25]

Hershberg, P. I., Goldfinger, S. E., Lemon, F. R., and Fessel, W. J. 1972. Medical record as index of quality of care. New. Eng. J. Med,. 286, 725--726.

[26]

Hiddema-Van De Wal, A., Smith, R. J., Van Der Werf, G. T., and Meyboom-De Jong, B. 2001. Towards improvement of the accuracy and completeness of medication registration with the use of an electronic medical record (EMR). Fam. Pract. 18, 288--291.

[27]

Humphreys, T., Shofer, F. S., Jacobson, S., Coutifaris, C., and Stemhagen, A. 1992. Preformatted charts improve documentation in the emergency department. Ann. Emerg. Med. 21, 534--540.

[28]

Jeans, W. D., Danton, R. M., and Kilburn, A. R. 1980. An assessment of a computerized reporting system (SIREP). British J. Radiol. 53, 421--427.

[29]

Johnson, K. B. and Cowan, J. 2002. Clictate: A computer-based documentation tool for guideline-based care. J. Med. Syst. 26, 47--60.

Digital Library

[30]

Johnson, K. B., Ravich, W. J., and Cowan, J. A., Jr. 2004. Brainstorming about next-generation computer-based documentation: an AMIA clinical working group survey. Int. J. Med. Inf. 73, 665--674.

[31]

Johnson, K. B., Serwint, J. R., Fagan, L. M., Thompson, R. E., and Wilson, M. H. 2005. Computer-based documentation: effect on parent and physician satisfaction during a pediatric health maintenance encounter. Arch. Pediatr. Adolesc. Med. 159, 250--254.

[32]

Kahn, C. E., Jr. and Huynh, P. N. 1996. Knowledge representation for platform-independent structured reporting. In Proceedings of the AMIA Annual Fall Symposium. 478--482.

[33]

Kiefer, V. F., Schwartz, R. J., and Jacobs, L. M. 1993. The effect of quality assurance on flight nurse documentation. Air Med. J. 1, 11--14.

[34]

Kuhn, K., Gaus, W., Wechsler, J. G., Janowitz, P., Tudyka, J., Kratzer, W., Swobodnik, W., and Ditschuneit, H. 1992. Structured reporting of medical findings: Evaluation of a system in gastroenterology. Methods Inf. Med. 31, 268--274.

[35]

Kuhn, K., Swobodnik, W., Johannes, R. S., Zemmler, T., Stange, E. F., Ditschuneit, H., and Classen, M. 1991. The quality of gastroenterological reports based on free text dictation: an evaluation in endoscopy and ultrasonography. Endoscopy 23, 262--264.

[36]

Ledley, R. S. 1966. Computer aids to medical diagnosis. J. Amer. Med. Assoc. 196, 933--943.

[37]

Lum, F., Schein, O., Schachat, A. P., Abbott, R. L., Hoskins, H. D., Jr., and Steinberg, E. P. 2000. Initial two years of experience with the AAO National Eyecare Outcomes Network (NEON) cataract surgery database. Ophthalmology 107, 691--697.

[38]

Madlon-Kay, D. J. 1998. Use of a structured encounter form to improve well-child care documentation. Arch. Fam. Med. 7, 480--483.

[39]

Marill, K. A., Gauharou, E. S., Nelson, B. K., Peterson, M. A., Curtis, R. L., and Gonzalez, M. R. 1999. Prospective, randomized trial of template-assisted versus undirected written recording of physician records in the emergency department. Ann. Emerg. Med. 33, 500--509.

[40]

Marshall, M., Campbell, C., Hacker, J., and Rowland, M. 2002. Quality Indicators for General Practice: A Practical Guide for Health Professionalsa and Managers. Oxford University Press, London.

[41]

McDonald, C. J. 1976. Protocol-based computer reminders, the quality of care and the non-perfectability of man. New. Eng. J. Med. 295, 1351--1355.

[42]

McDonald, C. J. 1997. The barriers to electronic medical record systems and how to overcome them. J. Amer. Med. Inf. Assoc 4, 213--221.

[43]

McDonald, C. J., Overhage, J. M., Dexter, P., Takesue, B. Y., and Dwyer, D. M. 1997. A framework for capturing clinical data sets from computerized sources. Ann. Intern. Med. 127, 675--682.

[44]

McDonald, C. J., Overhage, J. M., Tierney, W. M., Dexter, P. R., Martin, D. K., Suico, J. G., Zafar, A., Schadow, G., Blevins, L., Glazener, T., Meeks-Johnson, J., Lemmon, L., Warvel, J., Porterfield, B., Warvel, J., Cassidy, P., Lindbergh, D., Belsito, A., Tucker, M., Williams, B., and Wodniak, C. 1999. The Regenstrief Medical Record System: A quarter century experience. Int. J. Med. Inf. 54, 225--253.

[45]

McDonald, C. J., Tierney, W. M., Overhage, J. M., Martin, D. K., and Wilson, G. A. 1992. The Regenstrief Medical Record System: 20 years of experience in hospitals, clinics, and neighborhood health centers. MD Comput. 9, 206--217.

[46]

McDonald, C. J., Wilson, G. A., and McCabe, G. P., Jr. 1980. Physician response to computer reminders. J. Amer. Med. Assoc. 244, 1579--1581.

[47]

McGeary, M., Ford, M. A., McCutchen, M. R., and Barnes, D. K. 2007. A 21st century system for evaluating veterans for the disability benefits. IOM Board on Military and Veterans Health: Committee on Medical Evaluation of Veterans for Disability Compensation, Washington, D.C.

[48]

McKnight, L. K., Elkin, P. L., Ogren, P. V., and Chute, C. G. 1999. Barriers to the clinical implementation of compositionality. In Proceedings of the AMIA Symposium. 320--324.

[49]

Mehmood, K., Cherfi, S., and Comyn-Wattiau, I. 2009. Data quality through conceptual model quality -- reconciling researchers and practitioners through a customizable quality model. In Proceedings of the 14th International Conference on Information Quality (ICIQ'09). 61--74.

[50]

Modai, I. and Rabinowitz, J. 1993. Why and how to establish a computerized system for psychiatric case records. Hosp. Commun. Psych. 44, 1091--1095.

[51]

Nightingale, F. 1863. Notes on Hospitals. Longman, London.

[52]

O'Connor, A. E., Finnel, L., and Reid, J. 2001. Do preformatted charts improve doctors' documentation in a rural hospital emergency department&quest; A prospective trial. NZ Med. J. 114, 443--444.

[53]

Orman, L., Storey, V. C., and Wang, R. Y. 1996. Systems approaches to improving data quality. In Proceedings of the Conference on Information Quality. 117--126.

[54]

Ornstein, S. M., Jenkins, R. G., Lee, F. W., Sack, J. L., Lakier, E. I., Roskin, S. D., Wulfman, J. S., and Wriston, G. A. 1997. The computer-based patient record as a CQI tool in a family medicine center. Joint Comm. J. Qual. Improv. 23, 347--361.

[55]

Orr, K. 1998. Data quality and systems theory. Comm. ACM 41, 66--71.

Digital Library

[56]

Pierik, F. H., Van Ginneken, A. M., Dohle, G. R., Vreeburg, J. T., and Weber, R. F. 2000. The advantages of standardized evaluation of male infertility. Int. J. Androl. 23, 340--346.

[57]

Poon, A. D., Fagan, L. M., and Shortliffe, E. H. 1996. The PEN-Ivory project: Exploring user-interface design for the selection of items from large controlled vocabularies of medicine. J. Amer. Med. Inform. Assoc. 3, 168--183.

[58]

Powsner, S. M., Riely, C. A., Barwick, K. W., Morrow, J. S., and Miller, P. L. 1989. Automated bibliographic retrieval based on current topics in hepatology: hepatopix. Comput. Biomed. Res. 22, 552--564.

Digital Library

[59]

Rabinowitz, J., Modai, I., Valevski, A., Zemishlany, Z., and Mark, M. 1993. Benefits of a structured format for paper and computerized psychiatric case records. Hosp. Commun. Psych. 44, 1095--1097.

[60]

VA Office of the Inspector General. Review of state variances in VA disability compensation payments.

[61]

Rosenbloom, S. T., Crow, A. N., Blackford, J. U., and Johnson, K. B. 2006. Cognitive factors influencing perceptions of clinical documentation tools. J. Biomed. Inf.

Digital Library

[62]

Rosenbloom, S. T., Kiepek, W., Belletti, J., Adams, P., Shuxteau, K., Johnson, K. B., Elkin, P. L., and Shultz, E. K. 2004. Generating complex clinical documents using structured entry and reporting. MEDINFO'04, 683--687.

[63]

Rosenbloom, S. T., Miller, R. A., Johnson, K. B., Elkin, P. L., and Brown, S. H. 2006. Interface terminologies: facilitating direct entry of clinical data into electronic health record systems. J. Amer. Med. Inform. Assoc. 13, 277--288.

[64]

Safran, D. G., Kosinski, M., Tarlov, A. R., Rogers, W. H., Taira, D. H., Lieberman, N., and Ware, J. E. 1998. The Primary Care Assessment Survey: Tests of data quality and measurement performance. Med. Care 36, 728--739.

[65]

Sahlstedt, S., Adolfsson, H., Ehnfors, M., and Kallstrom, B. 1997. Nursing process documentation--Effects on workload and quality when using a computer program and a key word model for nursing documentation. Stud. Health Technol. Inf. 46, 330--336.

[66]

Schriger, D. L., Baraff, L. J., Rogers, W. H., and Cretin, S. 1997. Implementation of clinical guidelines using a computer charting system. Effect on the initial care of health care workers exposed to body fluids. J. Amer. Med. Assoc. 278, 1585--1590.

[67]

Schwartz, R. J., Boisoneau, D. and Jacobs, L. M. 1995. The quantity of cause-of-injury information documented on the medical record: an appeal for injury prevention. Acad. Emerg. Med. 2, 98--103.

[68]

Scott, J. T. 2007. Honoring the call to duty: Veterans' disability benefits in the 21st century. Disability Commission, Washington, D.C.

[69]

Seltzer, S. E., Kelly, P., Adams, D. F., Chiango, B. F., Viera, M. A., Fener, E., Hooton, S., Bannon-Rohrbach, S., Healy, C. D., Doubilet, P. M., and Holman, B. L. 1997. Expediting the turnaround of radiology reports in a teaching hospital setting. AJR Amer. J. Roentgenol 168, 889--893.

[70]

Solomon, D. H., Schaffer, J. L., Katz, J. N., Horsky, J., Burdick, E., Nadler, E., and Bates, D. W. 2000. Can history and physical examination be used as markers of quality&quest; An analysis of the initial visit note in musculoskeletal care. Med. Care 38, 383--391.

[71]

Stead, W. W., Hammond, W. E., and Estes, E. H. 1977. Evaluation of an audio mode of the automated medical history. Methods Inf. Med. 16, 20--23.

[72]

Stead, W. W., Heyman, A., Thompson, H. K., and Hammond, W. E. 1972. Computer-assisted interview of patients with functional headache. Arch. Intern. Med. 129, 950--955.

[73]

Stetson, P. D., Morrison, F. P., Bakken, S., and Johnson, S. B. 2008. Preliminary development of the physician documentation quality instrument. J. Amer. Med. Inform. Assoc. 15, 534--541.

[74]

Teo, S., Hanson, R., Van Asperen, P., Giles, H., Fasher, B., Davis, A. M., and Kristidis, P. 1995. Improving asthma documentation in a paediatric emergency department. J. Paediatr. Child Health 31, 130--133.

[75]

Tierney, W. M., Miller, M. E., Overhage, J. M., and McDonald, C. J. 1993. Physician inpatient order writing on microcomputer workstations. Effects on resource utilization. J. Amer. Med. Assoc. 269, 379--383.

[76]

Veterans' Disability Benefits. 2005. VA could enhance its progress in complying with court decision on disability criteria. United States Government Accountability Office, Washington, D.C.

[77]

Walters, B. C. and McNeill, I. 1990. Improving the record of patient assessment in the trauma room. J. Trauma 30, 398--409.

[78]

Weeks, W. B., Mills, P. D., Waldron, J., Brown, S. H., Speroff, T., and Coulson, L. R. 2003. A model for improving the quality and timeliness of compensation and pension examinations in VA facilities. J. Healthcare Manage. 48, 252--261; discussion, 262.

[79]

Weinger, M. B., Herndon, O. W., and Gaba, D. M. 1997. The effect of electronic record keeping and transesophageal echocardiography on task distribution, workload, and vigilance during cardiac anesthesia. Anesthesiology 87, 144--155; 129A--130A.

[80]

Winickoff, R. N., Coltin, K. L., Fleishman, S. J., and Barnett, G. O. 1986. Semiautomated reminder system for improving syphilis management. J. Gen. Intern. Med. 1, 78--84.

[81]

Wirtschafter, D. D., Scalise, M., Henke, C., and Gams, R. A. 1981. Do information systems improve the quality of clinical research&quest; Results of a randomized trial in a cooperative multi-institutional cancer group. Comput. Biomed. Res. 14, 78--90.

[82]

Wrenn, K., Rodewald, L., Lumb, E., and Slovis, C. 1993. The use of structured, complaint-specific patient encounter forms in the emergency department. Ann. Emerg. Med. 22, 805--812.

[83]

Zenni, E. A. and Robinson, T. N. 1996. Effects of structured encounter forms on pediatric house staff knowledge, parent satisfaction, and quality of care. A randomized, controlled trial. Arch. Pediatr. Adolesc. Med. 150, 975--980.

Cited By

Hakulinen JKeskinen TTurunen MSiltanen S(2021)Design Space for Voice-Based Professional ReportingMultimodal Technologies and Interaction10.3390/mti50100035:1(3)Online publication date: 11-Jan-2021
https://doi.org/10.3390/mti5010003
St-Maurice JBurns C(2017)An Exploratory Case Study to Understand Primary Care Users and Their Data Quality TradeoffsJournal of Data and Information Quality10.1145/30587508:3-4(1-24)Online publication date: 10-Jul-2017
https://dl.acm.org/doi/10.1145/3058750

Index Terms

Documentation quality and time costs: A randomized controlled trial of structured entry versus dictation

Recommendations

Effects of a Computer-based Nursing Documentation System on the Quality of Nursing Documentation

Nursing documentation is an important part of clinical documentation. However, documentation of the nursing process is frequently lacking quality. There are high expectations that computer support in nursing documentation will help improve documentation ...
Using GIS to Profile Health-Care Costs of VA Quality-Enhancement Research Initiative Diseases

The Health Services Research and Development (HSR&D) Service at the Department of Veterans Affairs (VA) Health Care System launched a Quality Enhancement Research Initiative (QUERI) in 1998. This study estimated health-care costs of nine diseases under ...
Microsoft Windows XP Resource Kit Documentation

Comments

Information & Contributors

Information

Published In

cover image Journal of Data and Information Quality

Journal of Data and Information Quality Volume 3, Issue 1

April 2012

54 pages

ISSN:1936-1955

EISSN:1936-1963

DOI:10.1145/2166788

Issue’s Table of Contents

Copyright © 2012 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 07 May 2012

Accepted: 01 January 2012

Revised: 01 November 2010

Received: 01 January 2010

Published in JDIQ Volume 3, Issue 1

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Qualifiers

Research-article
Research
Refereed

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

2
Total Citations
View Citations
537
Total Downloads

Downloads (Last 12 months)8
Downloads (Last 6 weeks)1

Reflects downloads up to 07 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Hakulinen JKeskinen TTurunen MSiltanen S(2021)Design Space for Voice-Based Professional ReportingMultimodal Technologies and Interaction10.3390/mti50100035:1(3)Online publication date: 11-Jan-2021
https://doi.org/10.3390/mti5010003
St-Maurice JBurns C(2017)An Exploratory Case Study to Understand Primary Care Users and Their Data Quality TradeoffsJournal of Data and Information Quality10.1145/30587508:3-4(1-24)Online publication date: 10-Jul-2017
https://dl.acm.org/doi/10.1145/3058750

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Issue’s Table of Contents