Keywords

1 Introduction

Sepsis occurs when the body’s response to an infection damages its own tissues and organs. It can lead to shock, multiple organ failures and death, especially if not recognized early and treated promptly. Every hour of delay of appropriate antibiotic therapy increases mortality by 5–10% [3]. Yet, to successfully diagnose sepsis is one of the greatest challenges in critical care. As the symptoms for sepsis are unspecific and similar to other conditions, it remains challenging for the clinician to identify the sepsis patients, especially at an early stage. Moreover, current diagnostic methods, including for example blood cultures, are impaired by a significant time-delay of 1–2 days [33] and/or low sensitivity of 30–40% [5]. This can result in a delay of appropriate patient care, which in turn can lead to a worsen condition for the patient with more severe complications and longer hospitalizations.

To enable early diagnosis of sepsis, research from three fronts are being conducted. One is the development of multimarker panels. Data mining techniques are being utilized for the selection of several subsets of combinations of biological markers and clinical data [26, 27], aiding earlier diagnosis, since the biological markers are monitored directly from the patient’s blood. The data mining will provide a classification score for the probability of sepsis as an outcome. This is to be applied, understood and interpreted by the healthcare system facilitating support for clinical decision making. Due to the complexity of sepsis diagnostics, it is imperative to include healthcare personnel in the data analysis process. Therefore, the second main effort focuses on methods for explaining and visualizing results from the data mining process, aiming to increase the interpretability and transparency of the sepsis diagnosis support system. The third focus concerns the more general development of clinical decision support systems (CDSSs) that, based on the collected patient data from physical exams, laboratory results etc., can provide early warnings to the healthcare professionals (see for instance [2]). These systems range from being purely rule-based to using techniques such as artificial neural networks, support vector machines and decision trees to create models suitable for sepsis diagnosis [1, 18].

The future of sepsis diagnosis research must, of course, join these efforts, yet they all suffer from their inherent difficulties. The presence of noisy, uncertain and non-linear data aggravates the diagnostic power of the analysis techniques as well as their interpretability. Moreover, CDSSs have not always been positively welcomed in the healthcare sector due to factors such as staff resistance [9, 19]. Reasons for the resistance are for example the skepticism regarding the benefits of using such support systems, the imposed changes of their way of working and the lack of transparency of the support system reasoning, i.e. that no information is provided regarding the sources, the strength of the evidence and the reliability of the results generated [9, 11, 16, 19]. Moreover, the inability of such systems to accommodate for clinician input has further been cited as a major reason for the resistance [17].

To facilitate a more easy adoption of a future sepsis diagnosis support system, semi-structured, in-depth interviews with four clinicians and one biomedical analyst have been performed to explore which decision support characteristics are vital for early, efficient and accurate diagnosis of sepsis. The participants were all experts of blood analyses within the field of severe infections and worked at a hospital and clinical laboratory in Sweden. The focus of the interviews was to identify the information needs of the healthcare professionals to make an informed decision as early as possible, the analysis strategies carried out, the challenges associated with diagnosing sepsis and the support tools used today. The results obtained are to be used in the future development of a digital sepsis diagnostic tool. Further, we contrast and compare out results with similar studies of CDSSs in order to contribute with general knowledge of how healthcare professionals can be supported in the process of diagnosing sepsis at an early stage.

The paper is structured as follows: Sect. 2 presents a general overview of CDSSs and guidelines for their implementation, together with research concerning support systems used for diagnosing and treating patients who suffer from sepsis. Section 3 summarizes the study, whereas Sect. 4 presents the results obtained. Section 5 offers a discussion of the work presented, whereas conclusions and directions for future research are presented in Sect. 6.

2 Background

According to Musen et al. [25], a clinical decision support system is any computer program designed to help healthcare professionals make clinical decisions. This broad definition thus comprises any computerized system that can aid its users to, for example, manage clinical information, highlight important information in medical records as well as provide patient-specific recommendations. Berner and La Lande [4] provide an important addition to this definition, arguing that a CDSS must provide guidance to the healthcare professionals at the point of care, thus excluding systems that only provide retrospective analyses of clinical data.

CDSSs can also be described by other means. For example, Metzger and MacDonald [23] have categorized CDSS in terms of when in the clinical process that the support system provides guidance, how active or passive the support system is, the ability of the system to provide adaptable support to the clinical situation, as well as how easy it is for the clinician to access the tool during his/her work flow. Yet another categorization is whether the system is stand-alone or a subsystem of more general electronic health records (EHRs), as well as if the CDSS is knowledge-based or non knowledge-based, i.e. that uses machine learning to generate recommendations [4].

Many of the CDSSs used today stem from research on expert systems, where the aim was to build software that could simulate human thinking through mapping signs, symptoms and laboratory results into probabilistic estimates of different diagnoses [24]. The trend nowadays is rather to develop support systems that assist the clinician in his/her decision making, and where the user is active in the decision making process [4].

2.1 Why and How to Implement CDSS?

Several studies have concluded that clinical decision support can improve clinician performance (see for instance [4, 12, 14, 21]). Positive effects concern, for example, the stronger adherence to relevant guidelines, the cost of care, the reduction of medication errors and decreased rates of potential redundant or inappropriate care [7, 10, 15]. However, a growing pool of research indicates that the anticipated positive effects are often unrealized, as well as that the impact of introducing the CDSS on the workflow of the clinicians has not been sufficiently evaluated [10, 16, 32]. For instance, in a survey conducted by Garg et al. [16], it was concluded that in the majority of cases included in their study, the introduction of a CDSS required more time and effort from the users compared with the previously used paper based methods.

Factors obstructing the positive effects of CDSSs are, for example, the failure of the practitioners to use the system, poor usability and integration into the users’ workflow, as well as user non-acceptance [16]. For example, positive results have been found in studies where the users were automatically using the system through tight integration with the already established computerized systems used, than when the users had to actively initiate the system [16]. Another factor that might have a negative impact on the acceptance of the users is the anticipated dependence on the systems, eroding the possibilities for independent decision making.

Based on the observed positive and negative effects of introducing CDSSs in the healthcare domain, several researchers have published guidelines for CDSS implementation. For example, Kawamoto et al. [20] examined factors associated with CDSS success across a variety of studies and found four critical factors:

  • provide alerts/reminders automatically as part of the workflow,

  • provide the suggestions at a time and location where the decisions are being made,

  • provide actionable recommendations and

  • computerize the entire process.

Berner and La Lande [4] also offer a set of general guidelines for the appropriate implementation and usage of CDSSs:

  • assure that users understand the limitations of the system,

  • assure that the knowledge base stems from reputable sources (from expert clinicians and/or clinical practice guidelines),

  • assure that the system is appropriate for the local site,

  • assure that the users are properly trained,

  • monitor the proper utilization of the installed CDSS and

  • assure that the knowledge base is monitored and maintained.

According to Chaudhry et al. [10] and Berner and La Lande [4], positive results of introducing a new CDSS are most often found in cases where there has been an incremental and local development during several years, led by researchers within the field, and where other computerized support systems are an accepted part of the work environment. Such prerequisites are often not possible to fulfill for most institutions, making it difficult to apply the same development and transition strategies showcased as promoting success. As such, further research is needed where studies of the effectiveness of CDSSs are in focus in order to identify additional design recommendations [4, 16].

2.2 CDSS for Sepsis Diagnosis

Due to the complexities associated with sepsis diagnosis and treatment, several researchers have proposed and implemented computerized support tools for such use. For example, Mathe et al. [22], present the Sepsis Treatment Enhanced through Electronic Protocolization (Steep) system. This system aids clinicians adhere to regulated treatment plans for sepsis as well as to monitor the patient’s status through visual means. The system fetches real-time patient data from the patient’s records, and the clinician is able to order medications and procedures through the tool. The system monitors specific lab and vital-signs abnormalities and provides alerts and treatment guidelines, both visually and through electronic messages to the healthcare team, when such values have been measured. However, this tool was developed as a stand-alone tool, requiring active activation of the clinicians and no evaluation of its appropriate usage is provided.

Other tools for diagnosing and classifying patients with sepsis are described in [2, 6, 28]. Amland and Hahn-Cover [2] describe an alert system integrated into the hospital environment and clinical workflow of the healthcare institution selected for their study. Through a text-based graphical interface, the sepsis criteria measured and their time stamp are displayed. The clinicians are thus provided with the rules applied by the system, as well as how severe the sepsis is. Yet, the study presented in [2] as well as in [6, 28] report struggles with designing the algorithms used for the sepsis diagnosis where high sensitivity in the diagnosis process are achieved at the cost of low specificity, setting the arena for potential usability problems associated with too many false positive alarms.

A web-based support system for sepsis treatment is presented in [31], where the healthcare practitioners are guided through the sepsis diagnosis process with the help of rule-based and data-driven logic algorithms, providing patient-adapted treatment instructions. Their research indicate that the healthcare practitioners were more inclined to adhere to the nationally and locally established guidelines for sepsis treatment as well as that their diagnostics improved when using the support system. Moreover, the number of antibiotic-free days increased, together with a shortened time until antibiotics were administered when needed when the system was used. However, no information is given regarding how the actual information was presented to the healthcare professionals, as well as how these users experienced the tool.

The above studies can all provide valuable guidance and inspiration when designing a future CDSS for sepsis diagnosis and treatment at the hospital and laboratories in focus of this study. However, as stated by Chaudhry et al. [10] and Berner and La Lande [4], one observed factor of success is the local development of the tool, where a tight incorporation of the work practices together with early and continuous user involvement can lay the foundation for greater user acceptance and a positive impact on the users’ performance.

3 Method

To extract guidelines for the local development of a tool capable of aiding its users to diagnose sepsis, in-depth, semi-structured interviews with five healthcare practitioners were conducted. Of these, one worked as a senior clinician in infectious diseases at a hospital in Sweden. Three worked as laboratory clinicians and one as a biomedical analyst at a clinical laboratory, performing chemical and biomedical analyses of blood samples. The interviewees had an average experience of 17.2 years within their respective field and the interviews took about an hour each to perform. Three were men, two were women. The interviewees were chosen due to their long experience of working with sepsis diagnosis at their respective care unit. They were also selected due to their different roles in the sepsis diagnosis process, where the clinicians and lab analysts work very closely to determine if a patient suffers from sepsis or not.

The hospital selected for the study had approximately 400 000 visiting patients during 2015, of whom around 40 000 were admitted during the same year. Around 8000 blood cultures are done each year at the clinical laboratory and of these close to 4000 patients are suspected to suffer from sepsis (no data is available regarding how many of these patients actually suffer from sepsis or not). All of the interviewees worked with different CDSSs and computerized tools to solve their working tasks. In terms of sepsis diagnosis and treatment, healthcare professionals in Sweden are expected to follow nationally established guidelines together with local protocols. These come in the form of checklists to use during the diagnosis process, as well as a procedure for determining if the patient has sepsis or not through if-else rules.

The questions asked were (translation from Swedish):

  • Describe your general working tasks.

  • Describe the actions you perform and decisions you make when you suspect that a patient suffers from sepsis.

    • How do you perform these tasks and make these decisions?

    • How do you communicate and collaborate with your colleagues in order to determine if a patient suffers from sepsis or not?

  • Do you use any kind of digital support to solve your working tasks?

    • How would you say that these aid you in your work?

    • Which input do you give to this system/which output can it provide to you? In which form?

  • Do you use any kind of digital support system to solve your working tasks related to sepsis diagnosis?

  • How do you think that the process of diagnosing sepsis could be enhanced in terms of speed and accuracy?

4 Results

Since the interviewees worked at three different care units (i.e. at the department of infectious diseases at the hospital as well as at the clinical chemical lab and the clinical microbiology lab), responsible for different parts of the sepsis diagnosis process, the results will be arranged using a time line of the patient’s healthcare process when sepsis is suspected. The results depict the healthcare professionals work tasks from when a patient arrives at the emergency ward, the support systems used and their ideas for the future development of a CDSS (or CDSS competent) to diagnose sepsis at an early stage that would suit well into their current workflow.

4.1 The Process of Detecting Sepsis

Emergency Ward

The ambulance personnel makes a first judgment how seriously ill the patient is and to which healthcare unit the patient should be directed to. If deemed urgent, the patient is sent to the emergency ward. When the patient arrives at the emergency ward, a nurse creates a digital emergency chart, incorporating all patient parameters measured by the ambulance personnel. This chart is continuously updated with data from new laboratory tests or vital-sign parameters such as the patient’s pulse, breathing, blood pressure, mental status and if the patient is in pain. The nurse or clinician further orders a set of routine tests if sepsis is suspected. These are regulated in the national guidelines and in the local hospital regulations. If many or some of the more important parameters indicate that the patient has a severe infection, blood will be withdrawn for blood culture and intravenous antibiotic will be provided immediately. But if there is deemed to be time without venturing the patient’s health, lab tests are ordered directly through the EHR system.

The EHR system visually represents data in tabular format. Clinician entries and lab results are displayed in text and numbers. Different tabs present data from different lab tests, and a clinician must often switch between different tabs to collect a full view of a patient’s status. There is a possibility to create a personalized view, collecting different lab results in one tab, yet the result is often that the clinician has to scroll through clinical parameter values that do not fit into the same view. If a measured parameter is deemed to fall outside a pre-determined threshold, this value is marked with red font. Yet, the clinician has to actively scroll down to the parameter of interest and look at the color of the value, i.e. there is no alarm or pop-up message available to aid the clinician. The clinician in infectious diseases interviewed argued that values not falling within the accepted interval should be visualized and interacted with in a better way than just by marking the text red in the text-based EHR system used. Moreover, s/he argued that the users of the system should be presented with an aggregated view of the test results to better grasp the complete status of the patient. Further, the clinician in infectious diseases argued that some decision support rules should be incorporated into the EHR system that set off an alarm when a clinical parameter value/the aggregation of clinical parameter values that might indicate sepsis have been fulfilled. Since few clinicians are infection specialists, sepsis can quite easily be missed for something else - for example, the signs of a heart attack might as well be due to sepsis. Such rules could minimize the risk of missing such important findings in the data. According to the clinician, this should preferably come in the form of a pop-out dialogue box, presenting the important findings in the data, what it can indicate as well as what the next step in the treatment process should be.

The clinician in infectious diseases further stressed the importance of being able to compare the measured clinical parameters of each patient, and to detecting trends and possible effects of recommended treatments, functions that are not supported by the EHR system used today. Such functionality could have a possible effect on the clinician’s decision making, where s/he can easily see the status of a patient in one and the same view.

The clinician in infectious diseases interviewed argued for the importance of developing decision support systems of this kind for ensuring patient safety. S/he stressed the importance of the system having a supporting role, indicating important findings and recommending actions to make sure that the patients receive treatment fast. Important is also that the users of such system receives appropriate training and is able to interact with the system in a good way. Also important is the fact that the clinicians know the rules and methods behind the recommendations, that they follow the national and local guidelines, and that it makes its users aware of interesting findings.

Clinical Chemical and Microbiology Laboratory

When lab tests are ordered by the hospital personnel, blood samples are withdrawn from the patient and delivered to the clinical lab either manually or by using the tube system at the current hospital. Depending on the tests ordered, either the clinical chemical lab or the clinical microbiology lab professionals are in charge of the tests. At the clinical chemical lab, most of the analysis is done automatically. The whole process of receiving blood samples in tubes, to the analysis and reporting of the results back to the clinician is today fully automatic. For all tests, there are reference intervals indicating normal/abnormal values. These threshold values are updated if new research or guidelines are to be executed. If a measured value does not fall within this reference interval, an alarm is set off and manual analysis is carried out. In order for this process to work, meticulous controls of the clinical lab machines are carried out. Despite the high level of automation, the chemical analysts interviewed claimed that as they work as consultants to the clinicians, it is of utmost importance that they have knowledge of what the different tests indicate, as well as that they are able to perform manual analyses in the case of, for example, machine malfunction or if a test result is inconclusive.

Manual analyses at the clinical chemical lab are mostly performed in the form of protein profile analyses that can determine which type of bacteria is causing the sepsis, however such analyses are not common (in about 1% of the cases). Here, images of proteins are manually analyzed and the results are reported back to the clinician via the EHR and through oral communication with the clinician in charge of the patient’s care.

At the clinical microbiology lab, more manual work is conducted. A first analysis is performed using a dedicated machine that every 10th minute investigates if bacteria have started to grow in the blood cultures or not. When the bacteria has reached a certain density of growth, an audio and visual alarm is set off. The blood cultures are then analyzed manually in a microscope where the analyst is able to make a first judgment regarding the type of bacteria found. Here, the analyst uses his/her expert knowledge and looks for the color, shape, groupings etc. of the bacteria found. If more specific results are sought for, blood culturing are followed by subculture on agar plates for subsequent phenotypic identification. Manual and automatic pattern analysis is made of the bacteria in order to detect patterns, aggregate findings, etc. to determine which species of bacteria that has been found. Yet this process takes several hours, time often not available in the case of sepsis.

The lab analysts argued for an appropriate level of CDSS automation, where the analyst is still kept in the loop. As such, the support systems should be a natural component of the workflow, acting as support and have an advisory role, aiding the analysts to recall important parameters and values. Such support would also aid the analysts if manual control is needed, delimiting the risk of alienating them from the tasks conducted. Moreover, the analysts argued that the CDSS should be designed to delimit the tedious manual workload of, for example, updating the patient’s EHR with lab data, as has been the case with some support systems used. Further, the system should provide appropriate alarms, based on national and local guidelines, where the reason for the alarm is explicitly presented, i.e. which parameter threshold values have been met? Which evidence point towards a certain diagnosis, and which do not?

Despite the increasing level of sophisticated automation at the laboratory, one of the laboratory clinicians argued for constant and increased communication with the hospital clinicians, together with continuous updates of the patient’s status to keep the patient in focus, not the numbers in a lab report. This was also stressed by the microbiological analyst participating in the study, who argued that a closer collaboration between the clinical lab personnel and the hospital clinicians could improve the accuracy of the diagnoses made. Due to their specialized knowledge of different bacteria and their antibiotic resistance, additional, more specified, information could be provided to the hospital clinicians. However, such collaboration would demand that also the laboratory personnel has access to the patient EHR data, which is not the case today. Moreover, since most of the lab results are automatically sent to the hospital clinicians when ready, the clinicians can receive many different lab reports, which sometimes contain inconclusive results. If incrementally aggregating the lab results, the lab personnel could use their expertise to aid the hospital clinicians to make a more accurate decision regarding the patient’s treatment.

5 Discussion

Apparent during the interviews was the fact that the clinicians and lab analysts have to deal with large amounts of data to solve their tasks. For the hospital clinicians, measured parameters must be manually processed, and the national and local guidelines remembered, often during time-critical and stressful situations. Moreover, since the symptoms of sepsis are quite general, the clinicians must have such guidelines in mind even when sepsis is not their main hypothesis regarding the patient’s status. Junior clinicians might follow the strategy to order every test available, resulting in even more, and perhaps irrelevant, data to include in the analysis process, thus aggravating the clinician’s decision making. Additionally, the number of laboratory tests ordered at hospitals often increase in a much faster pace than the recruitment of clinicians, adding to their workload and feeding the risk of overlooking important lab results [8].

According to Stadler et al. [30], the increasing usage of EHRs has lead to a dramatic increase and availability of digital healthcare data. Such data, presented in a comprehensive manner, can aid a clinician to get a quick view of the patient’s current and past status. If not, the sheer amount of data feeds the possibility of overlooking or misinterpreting the data. As further argued by Stadler et al. [30], the incorporation of interactive visualizations within the EHR systems should be investigated, where for example dashboards can be used to convey the information contained within the EHR. To incorporate visualizations of patient vital-sign parameters, individual lab results, together with an aggregation of the patient data to create a comprehensive view of the patient’s status could provide a basis for the support needed in order to diagnose sepsis at an early stage. Such information should not only be available to the hospital clinicians, but also to the lab personnel, enabling them to provide their expertise into the decision making process. Yet, even though the usage of dashboards as analytic tools is growing, their use with EHR data is still in its infancy [30].

The interviewees were all accustomed to using computerized support tools in their work and saw their increasingly important role within the healthcare sector as an information and support provider. The clinician in infectious diseases stressed the importance of using CDSSs as support in the decision making process, rather than as a means of performing analytical tasks fully automatically. Such support should come in the form of process and action advice, together with highlighting or through other means communicating important information based on an expert knowledge-base relevant for the tasks at hand. A commonly referenced description of different levels of automation is the one proposed by Sheridan and Verplank [29] where ten levels describe the task allocation between the human and the automation (from low to high). As stated by Cummings [13], higher levels of automation is often the best solution when automating tasks that require no flexibility in decision-making and with a low probability of system failure. Yet, the higher levels might not be suitable in time-critical domains where there are many external and changing constraints due to the possibility of imperfect and unreliable automation. From the scale, the CDSS to be used for the clinician’s diagnosis of sepsis should offer a set of decision/action alternatives, suggest the most probable ailment, and execute suggestions such as lab orders if the clinician approves to. Of course, in the lab, the automation can be higher, implying that the automation executes its tasks automatically, and only informs the analysts if there is something wrong with the test, or if the parameter values are out of range, however always informing its users of its knowledge-base and rules. As such, the developers of the CDSS should always have the importance of system transparency in mind.

In addition to the more general guidelines presented in Sect. 2.1, this study has resulted in the identification of guidelines for the development of a CDSS for sepsis diagnosis:

  • adjust the level of automation in accordance with the healthcare practitioners’ tasks and workflow,

  • visually aggregate the information regarding the patient’s status in the EHR, possibly through the usage of dashboards,

  • enable interactive means of investigating the patient data,

  • provide visual and audio warnings to the healthcare practitioners when an important parameter value/the aggregation of parameter values falls outside the reference interval,

  • enable a close collaboration between the lab personnel and the hospital clinicians and

  • enable the healthcare practitioners to inspect the knowledge base of the CDSS and the evidence supporting its recommendations.

5.1 Limitations of the Study

The results obtained from the interviews performed as well as the literature study made should be used as input to a second follow-up study where additional practitioners from the hospital and lab departments are represented. Since most of the manual decision making is performed by the hospital clinicians, additional interviews and observations must be performed together with this particular user group. A broader literature study of CDSSs used for other diagnoses and treatments should also be performed to identify additional important guidelines for the development of the support tool for sepsis diagnosis.

6 Conclusions and Future Work

Research for continuously making the sepsis diagnosis process as fast as possible is vital. More and more demands are put on healthcare practitioners, where increased effectiveness and improved patient care is required. The amount of data has increased in a much faster pace than the number of clinicians, increasing the risk of missing important analytical results, even if the systems used demand electronic verification. This study has focused on extracting general guidelines to be applied during the development process of a CDSS to be used for sepsis diagnosis. A follow-up study with additional participants from the healthcare domain is planned to extract further requirements to be used when implementing a first prototype of the CDSS. Additional work is also needed in terms of investigating how the analysis results from new lab tests should be visualized and interacted with to ensure their transparency.