Use of a remote clinical decision support service for a multicenter trial to implement prediction rules for children with minor blunt head trauma

https://doi.org/10.1016/j.ijmedinf.2015.12.002Get rights and content

Highlights

  • We describe use of CDS services in a trial of blunt head trauma prediction rules.

  • We evaluate the execution characteristics of the system at two emergency departments.

  • The CDS service was the source of recommendations in 60% of real-time workflows.

  • The CDS service was the source of recommendations in all near-real-time workflows.

  • Evolving CDS services facilitate sharing of CDS in selected clinical settings.

Abstract

Objective

To evaluate the architecture, integration requirements, and execution characteristics of a remote clinical decision support (CDS) service used in a multicenter clinical trial. The trial tested the efficacy of implementing brain injury prediction rules for children with minor blunt head trauma.

Materials and Methods

We integrated the Epic® electronic health record (EHR) with the Enterprise Clinical Rules Service (ECRS), a web-based CDS service, at two emergency departments. Patterns of CDS review included either a delayed, near-real-time review, where the physician viewed CDS recommendations generated by the nursing assessment, or a real-time review, where the physician viewed recommendations generated by their own documentation. A backstopping, vendor-based CDS triggered with zero delay when no recommendation was available in the EHR from the web-service. We assessed the execution characteristics of the integrated system and the source of the generated recommendations viewed by physicians.

Results

The ECRS mean execution time was 0.74  ± 0.72 s. Overall execution time was substantially different at the two sites, with mean total transaction times of 19.67 and 3.99 s. Of 1930 analyzed transactions from the two sites, 60% (310/521) of all physician documentation-initiated recommendations and 99% (1390/1409) of all nurse documentation-initiated recommendations originated from the remote web service.

Discussion

The remote CDS system was the source of recommendations in more than half of the real-time cases and virtually all the near-real-time cases. Comparisons are limited by allowable variation in user workflow and resolution of the EHR clock.

Conclusion

With maturation and adoption of standards for CDS services, remote CDS shows promise to decrease time-to-trial for multicenter evaluations of candidate decision support interventions.

Introduction

Minor head trauma is a common presenting condition of children seen in emergency departments (EDs). Of approximately 600,000 children younger than 18 years presenting to EDs annually in the United States for blunt head trauma, more than 95% is minor (defined by Glasgow Coma Scale [GCS] scores of 14–15), with a very low associated prevalence of clinically important traumatic brain injuries (ciTBIs; here defined as a death, neurosurgical intervention, intubation for more than 24 h, or hospital admission for 2 or more nights due to the head trauma in association with a positive cranial computed tomography [CT] scan) [1], [2], [3], [4]. However, because the use of ionizing radiation in children is associated with an increased lifetime risk of lethal malignancy of approximately 0.15%, cranial CT scans should be used judiciously in children with minor head trauma [5], [6], [7], [8]. In order to facilitate clinician decision-making and balance the need to identify ciTBIs with the risk of malignancy from CT scans, the Pediatric Emergency Care Applied Research Network (PECARN) derived and validated prediction rules for children younger than two years and children 2 up to their 18th birthday that identify those at very low risk of ciTBIs who do not require CT scans, based on a series of patient history and physical examination findings (Fig. 1) [4].

Clinical decision support (CDS) based on the PECARN prediction rules could deliver personalized imaging recommendations based on the risks of ciTBI to clinicians at the point-of-care in real-time. PECARN provides an in vivo laboratory in which to evaluate the effectiveness of these prediction rules using computer-based CDS, with 17 of 18 sites having electronic health records (EHR) with CDS capabilities. However, we have previously described the sociotechnical challenges involved in creating a successful CDS intervention in the ED setting across multiple sites, including technical complexity, human factors, and organizational and process dimensions [9]. The implementation of a CDS intervention for a multicenter evaluation is complicated by heterogeneous EHRs, differing versions/configurations of a single vendor’s EHR, varying technical capacity to conduct clinical research in an EHR production environment, and varying technical capacity of an institutional informatics staff to implement a CDS intervention. We describe one method to potentially decrease the complexity of implementing CDS interventions across multiple sites through the use of a novel remote CDS service.

Section snippets

Objective

Our objective was to evaluate the architecture, integration requirements, and execution characteristics of a remote CDS service used in a multicenter clinical trial, testing the effectiveness of an implementation of the PECARN traumatic brain injury (TBI) prediction rules for children with minor blunt head trauma. Researchers have emphasized the importance of characterizing these non-functional requirements of decision support systems, in addition to the outcomes they produce [10], [11], [12],

Traumatic brain injury clinical decision support trial

The PECARN TBI prediction rule CDS trial was a multicenter clinical trial that included thirteen EDs either in the PECARN or in Kaiser Permanente’s Clinical Research in Emergency Services and Treatments (CREST) network. Inclusion for sites receiving the CDS intervention was limited to those using the Epic® EHR (Epic Systems, Verona, WI) in order to minimize the complexity of the infrastructure and not introduce additional variation in implementation of the CDS. A total of ten sites implemented

Unified knowledge specification

We analyzed the data from both the original PECARN prediction rule manuscript and raw data in order to create the knowledge specification [4]. The knowledge specification captured the logic needed to evaluate the risk of ciTBI for patients in the two age groups in a spreadsheet format. From the data underlying the PECARN prediction rules, patients who were not in the very low risk group for ciTBI could be broadly categorized into two other risk groups based on specific clinical findings. For

Discussion

We have described the architecture, integration requirements, and performance characteristics of a remote CDS system designed to provide CT recommendations to physicians in the ED setting for the management of children with blunt head trauma. We successfully implemented two moderately complex prediction rules, taking patient history and physical examination data from the EHR and returning the risk of the patient having a clinically important TBI, along with recommendations regarding CT use. In

Conclusion

We have comprehensively evaluated the architecture, integration requirements, and execution characteristics of a remote CDS service in a multisite trial studying the effectiveness of implementation of the PECARN TBI prediction rules. In the ED workflow, most recommendations originated from the remote system when results were sought in less than one minute, and virtually all recommendations originated from the remote system in longer, near-real-time workflows. EHR adoption of evolving standards

Conflicts of interest

None.

Funding

American Recovery and Reinvestment Act—Office of the Secretary (ARRA OS): Grant #S02MC19289-01-00. PECARN is supported in part by the Health Resources and Services Administration (HRSA), Maternal and Child Health Bureau (MCHB), Emergency Medical Services for Children (EMSC) Network Development Demonstration Program under cooperative agreements U03MC00008, U03MC00001, U03MC00003, U03MC00006, U03MC00007, U03MC22684, and U03MC22685.

Author contributorship

The information or content and conclusions are those of the author and should not be construed as the official position or policy of, nor should any endorsements be inferred by HRSA, HHS or the U.S. Government.

All authors meet the ICMJE criteria for authorship. Contributions include the following:

HG: conception and design, analysis and interpretation of data, drafting the article and final approval of the version to be published. HG is the guarantor.

MP: conception and design, content

Acknowledgments

The authors acknowledge the work of the Clinical Decision Support Consortium (CDSC), led by Dr. Blackford Middleton and supported under a contract with the Agency for Healthcare Research and Quality (AHRQ), in the development of the four-layer knowledge specification methodology and ECRS. We also acknowledge the efforts of Sirina Lu at Nationwide Children’s Hospital, Richard Boyer at Partners HealthCare System, John Bear, Nola McDougall, and Mike Schmit at Children’s Hospital Colorado, Vickie

References (33)

  • M. Faul et al.

    Traumatic Brain Injury in the United States: Emergency Department Visits, Hospitalizations, and Deaths. Atlanta (GA): Centers for Disease Control and Prevention

    (2010)
  • D.P. Frush et al.

    The ALARA concept in pediatric imaging: building bridges between radiology and emergency medicine: consensus conference on imaging safety and quality for children in the emergency setting, Feb. 23–24, 2008, Orlando, FL—executive summary

    Pediatr. Radiol.

    (2008)
  • D.L. Miglioretti et al.

    The use of computed tomography in pediatrics and the associated radiation exposure and estimated cancer risk

    JAMA Pediatr.

    (2013)
  • J.D. Mathews et al.

    Cancer risk in 680,000 people exposed to computed tomography scans in childhood or adolescence: data linkage study of 11 million Australians

    BMJ

    (2013)
  • C. Catley et al.

    Software performance engineering of a web service-based clinical decision support infrastructure

  • V. Stantchev

    Performance evaluation of cloud computing offerings

    Proceedings of the 2009 Third International Conference on Advanced Engineering Computing and Applications in Sciences: IEEE Computer Society

    (2009)
  • Cited by (0)

    View full text