Use of a remote clinical decision support service for a multicenter trial to implement prediction rules for children with minor blunt head trauma
Introduction
Minor head trauma is a common presenting condition of children seen in emergency departments (EDs). Of approximately 600,000 children younger than 18 years presenting to EDs annually in the United States for blunt head trauma, more than 95% is minor (defined by Glasgow Coma Scale [GCS] scores of 14–15), with a very low associated prevalence of clinically important traumatic brain injuries (ciTBIs; here defined as a death, neurosurgical intervention, intubation for more than 24 h, or hospital admission for 2 or more nights due to the head trauma in association with a positive cranial computed tomography [CT] scan) [1], [2], [3], [4]. However, because the use of ionizing radiation in children is associated with an increased lifetime risk of lethal malignancy of approximately 0.15%, cranial CT scans should be used judiciously in children with minor head trauma [5], [6], [7], [8]. In order to facilitate clinician decision-making and balance the need to identify ciTBIs with the risk of malignancy from CT scans, the Pediatric Emergency Care Applied Research Network (PECARN) derived and validated prediction rules for children younger than two years and children 2 up to their 18th birthday that identify those at very low risk of ciTBIs who do not require CT scans, based on a series of patient history and physical examination findings (Fig. 1) [4].
Clinical decision support (CDS) based on the PECARN prediction rules could deliver personalized imaging recommendations based on the risks of ciTBI to clinicians at the point-of-care in real-time. PECARN provides an in vivo laboratory in which to evaluate the effectiveness of these prediction rules using computer-based CDS, with 17 of 18 sites having electronic health records (EHR) with CDS capabilities. However, we have previously described the sociotechnical challenges involved in creating a successful CDS intervention in the ED setting across multiple sites, including technical complexity, human factors, and organizational and process dimensions [9]. The implementation of a CDS intervention for a multicenter evaluation is complicated by heterogeneous EHRs, differing versions/configurations of a single vendor’s EHR, varying technical capacity to conduct clinical research in an EHR production environment, and varying technical capacity of an institutional informatics staff to implement a CDS intervention. We describe one method to potentially decrease the complexity of implementing CDS interventions across multiple sites through the use of a novel remote CDS service.
Section snippets
Objective
Our objective was to evaluate the architecture, integration requirements, and execution characteristics of a remote CDS service used in a multicenter clinical trial, testing the effectiveness of an implementation of the PECARN traumatic brain injury (TBI) prediction rules for children with minor blunt head trauma. Researchers have emphasized the importance of characterizing these non-functional requirements of decision support systems, in addition to the outcomes they produce [10], [11], [12],
Traumatic brain injury clinical decision support trial
The PECARN TBI prediction rule CDS trial was a multicenter clinical trial that included thirteen EDs either in the PECARN or in Kaiser Permanente’s Clinical Research in Emergency Services and Treatments (CREST) network. Inclusion for sites receiving the CDS intervention was limited to those using the Epic® EHR (Epic Systems, Verona, WI) in order to minimize the complexity of the infrastructure and not introduce additional variation in implementation of the CDS. A total of ten sites implemented
Unified knowledge specification
We analyzed the data from both the original PECARN prediction rule manuscript and raw data in order to create the knowledge specification [4]. The knowledge specification captured the logic needed to evaluate the risk of ciTBI for patients in the two age groups in a spreadsheet format. From the data underlying the PECARN prediction rules, patients who were not in the very low risk group for ciTBI could be broadly categorized into two other risk groups based on specific clinical findings. For
Discussion
We have described the architecture, integration requirements, and performance characteristics of a remote CDS system designed to provide CT recommendations to physicians in the ED setting for the management of children with blunt head trauma. We successfully implemented two moderately complex prediction rules, taking patient history and physical examination data from the EHR and returning the risk of the patient having a clinically important TBI, along with recommendations regarding CT use. In
Conclusion
We have comprehensively evaluated the architecture, integration requirements, and execution characteristics of a remote CDS service in a multisite trial studying the effectiveness of implementation of the PECARN TBI prediction rules. In the ED workflow, most recommendations originated from the remote system when results were sought in less than one minute, and virtually all recommendations originated from the remote system in longer, near-real-time workflows. EHR adoption of evolving standards
Conflicts of interest
None.
Funding
American Recovery and Reinvestment Act—Office of the Secretary (ARRA OS): Grant #S02MC19289-01-00. PECARN is supported in part by the Health Resources and Services Administration (HRSA), Maternal and Child Health Bureau (MCHB), Emergency Medical Services for Children (EMSC) Network Development Demonstration Program under cooperative agreements U03MC00008, U03MC00001, U03MC00003, U03MC00006, U03MC00007, U03MC22684, and U03MC22685.
Author contributorship
The information or content and conclusions are those of the author and should not be construed as the official position or policy of, nor should any endorsements be inferred by HRSA, HHS or the U.S. Government.
All authors meet the ICMJE criteria for authorship. Contributions include the following:
HG: conception and design, analysis and interpretation of data, drafting the article and final approval of the version to be published. HG is the guarantor.
MP: conception and design, content
Acknowledgments
The authors acknowledge the work of the Clinical Decision Support Consortium (CDSC), led by Dr. Blackford Middleton and supported under a contract with the Agency for Healthcare Research and Quality (AHRQ), in the development of the four-layer knowledge specification methodology and ECRS. We also acknowledge the efforts of Sirina Lu at Nationwide Children’s Hospital, Richard Boyer at Partners HealthCare System, John Bear, Nola McDougall, and Mike Schmit at Children’s Hospital Colorado, Vickie
References (33)
- et al.
Identification of children at very low risk of clinically-important brain injuries after head trauma: a prospective cohort study
Lancet
(2009) - et al.
Radiation exposure from CT scans in childhood and subsequent risk of leukaemia and brain tumours: a retrospective cohort study
Lancet
(2012) - et al.
Informing the design of clinical decision support services for evaluation of children with minor blunt head trauma in the emergency department: a sociotechnical analysis
J. Biomed. Inform.
(2013) - et al.
A multiple-criteria framework for evaluation of decision support systems
Omega
(2004) - et al.
Lessons learned from implementing service-oriented clinical decision support at four sites: a qualitative study
Int. J. Med. Inform.
(2015) - et al.
A framework and model for evaluating clinical decision support architectures
J. Biomed. Inform.
(2008) - et al.
A pilot study of distributed knowledge management and clinical decision support in the cloud
Artif. Intell. Med.
(2013) - et al.
HL7 clinical document architecture, release 2
J. Am. Med. Inform. Assoc.
(2006) - et al.
Incidence, risk factors and prevention of mild traumatic brain injury: results of the WHO collaborating centre task force on mild traumatic brain injury
J. Rehabil. Med.
(2004) - et al.
Derivation of the children’s head injury algorithm for the prediction of important clinical events decision rule for head injury in children
Arch. Dis. Child.
(2006)