skip to main content
research-article
Open access

IDE4ICDS: A Human-Centric and Model-Driven Proposal to Improve the Digitization of Clinical Practice Guideline

Published: 27 September 2024 Publication History

Abstract

Clinical practice guidelines (CPGs) are a formalization of specific clinical knowledge that states the best evidence-based clinical practices for treating pathologies. However, CPGs are limited because they are usually expressed as text. This gives rise to a certain level of ambiguity, subjective interpretation of the actions to be performed, and variability in clinical practice by different health professionals facing the same circumstances. The inherent complexity of CPGs is also a challenge for software engineers designing, developing, and maintaining software systems and clinical decision support system to manage and digitize them. This challenge stems from the need to evolve CPGs and design software systems capable of allowing their evolution. This paper proposes a model-driven, human-centric and tool-supported framework (called IDE4ICDS) for improving digitisation of CPG in practical environments. This framework is designed from a human-centric perspective to be used by mixed teams of clinicians and software engineers. It was also validated with the type 2 diabetes mellitus CPG in the Andalusian Public Health System (Spain) involving 89 patients and obtaining a kappa-based analysis. The recommendations were acceptable (0.61–0.80) with a total kappa index of 0.701, leading to the conclusion that the proposal provided appropriate recommendations for each patient.

1 Introduction

The origins of Clinical Practice Guidelines (CPGs) are to be found in the French «Médicine d’observation» movement [108] in the mid-nineteenth century. Proponents of that movement like Pierre C. Alexander Louis, Xavier Bichat, and François Magendie considered that healthcare professionals should not apply clinical methods based solely on their own personal experience and appreciation of their patients’ behavior, but that clinical practice should be based on research results with quantifiable conclusions [50]. Since the nineteenth century, this approach has evolved, improving clinical practice with techniques like randomized clinical trials (1948) [23] to perform scientific analyses based on statistical methods and clinical epidemiology. The new technique was accepted and improved in clinical practice after the publication in 1992 of a study by the Evidence-Based Medicine Working Group (Canada) [46] which demonstrated the effectiveness of evidence-based methods in improving the treatment of clinical pathologies. In this regard, it can be concluded that good clinical practice and healthcare depend on the translation of empirical evidence into CPGs [89]. At present, the concept of CPGs can be defined as «statements that include healthcare processes, clinical rules and recommendations intended to optimize patient care that are informed by a systematic review of evidence and an assessment of the benefits and harms of alternative care options» [58].
A well-defined CPG is beneficial to both healthcare professionals and patients [59, 62, 92]. For example: it makes it possible to reduce variability during clinical practice and improve the quality of clinical professionals’ performance and decision-making, it pools clinical knowledge obtained applying formal techniques that guarantee clinical and systematic validations of different clinical scenarios, and it facilitates effective, reliable interventions based on empirical evidence.
There now exist a number of different public repositories (US National Guideline Clearinghouse,1 Scottish Intercollegiate Guidelines Network2 and Wiley Cochrane Library,3 etc.) containing CPGs published by international organizations for numerous clinical domains (e.g., oncology, nutrition, cardiology, etc.), but these CPGs are not widely used in day-to-day clinical practice.
Although clinical guidelines can be considered as mechanisms for improving patient treatment, improving the performance of health professionals, and reducing variability in clinical practice [59], some researchers have identified factors which hinder their adoption by health professionals. For example, Lugtenberg et al. [63] conducted a qualitative study involving more than 30 professionals from different clinical organizations and identified limitations commonly experienced by the participants: lack of applicability, lack of evidence, lack of flexibility due to organizational restrictions, lack of clinical knowledge about the recommendations offered by each CPG and ambiguity in how those recommendations are described. Cabana et al. [17] also identified similar barriers associated with lack of awareness about the existence of guidelines, lack of consensus among health professionals and their institutions, perceived interference with the criteria of the healthcare professional, lack of understanding of clinical rules and recommendations, and the inherent difficulty of getting health professionals to change their habits.
Most of the above-mentioned factors restricting the adoption of clinical guidelines could be mitigated using software systems which allow CPG management to be fully digitized [7, 76] (from design or definition through to execution, evolution, and maintenance). However, one of the most recurrent barriers to the more widespread adoption of guidelines is the ambiguity inherent in their statements due to the fact that they are usually expressed in text format [17, 63].
This ambiguity affects both health professionals, who must daily apply and consider clinical recommendations, and software engineers, who digitize CPG and develop expert systems and Clinical Decision Support Systems (CDSSs) for CPG management. The software engineers also face an additional challenge in that they have to ensure the automatic maintainability of CPGs. This kind of system is usually developed ad hoc for clinical institutions because each institution has its own medical committees that decide how clinical guidelines should be used by their health professionals. And when public agencies publish new versions of CPGs, these must also be updated in each CDSS. In this regard, the development and maintenance of CDSSs are hard, complex, and expensive tasks. Other CPG-related challenges include the possible hindering of interoperability between different organizations, lack of standardization (since each hospital may decide to evolve or not to evolve the guidelines under execution in its systems), and the growing variability of clinical practice among professionals and even between hospitals.
Moreover, mechanisms must be researched, proposed, and defined to ensure the correct and successful execution and management of CPGs in order to reduce the development costs of CDSSs and reduce the degree of variability with which CPGs are applied for different patients in similar medical situations. It is important to mention that, although these systems may be correctly defined, the frequency of CPG evolution means that solutions must be provided regarding other essential aspects related to CPG maintenance and traceability. Managing change in an appropriate manner, maintaining traceability between the definition and implementation of CPGs, and achieving effective continuous improvement therefore become key tasks within health organizations.
The Contribution of the Paper. The final purpose of our work is to propose a model-driven and tool-supported framework to cover a set of clinical challenge of dealing with CPGs and a set of challenges and motivation for software engineers. From the clinical perspective, our proposal helps to: (i) facilitate the human-centered digitisation of CPGs within a continuous improvement lifecycle (from their definition to their execution, monitoring and maintenance); and (ii) reduce the variability of clinical practice when health professional carries out the follow-up of the pathology of their patients. From the software engineer perspective, our proposal looks for: (i) offering tools for automatically improving the definition and development of CDSS; and (ii) using the power of user-centered principles to offer a very power tools for automation but also very friendly to be used for non-technician people. In this sense, an interdisciplinary team of engineers and healthcare professionals were involved in our proposal to address these challenges from both perspectives, including a human-centric approach to consider the interaction between people and information technology (IT) tools [47] based on scientific evidence from CPGs. Specifically, it has been essential to take into account during the design and development of the software a human-centric approach to ensure the adoption of the developed tools in clinical practice.
For this purpose, this paper follows the Design Science Research Methodology (DSRM), formulating the following hypotheses: (i) «It is possible to successfully improve the design and development of CDSS with support for the digitisation of CPG through user-friendly, human-centric and model-based solutions»; and (ii) «It is possible to define a model-based language that contains concepts related to the definition, execution, orchestration,4 and monitoring of CPGs».
In this context, our proposal (called IDE4ICDS; Integrated Development Environment (IDE) for Improving Clinical Decision Support based on Clinical Guidelines (ICDS)) integrates our own CDSS and Model-Driven Engineering (MDE) [96] mechanisms, aiming to ensure the systematization, version compatibility, adaptability and interoperability needed to solve the above problems. Besides, our model-driven proposal seeks to simplify the understanding of a new system through abstract and graphical modelling [48], addressing technological heterogeneity and fostering productivity and rapid prototyping from a human-centered perspective. The main features of the IDE4ICDS platform are: (i) centralization, i.e., a single nucleus of information is stored and traceability is maintained between all the components of a CPG; (ii) integration, i.e., a single platform (IDE4ICDS) provides modules for defining, executing, and monitoring CPGs; and (iii) collaboration, i.e., if a health committee wishes to improve or evolve a CPG, they can do it intuitively and in a user-friendly fashion, and the resulting modification can subsequently be used by another health organization. These requirements have been included by incorporation of human centric-issues throughout the MDE process lifecycle, whose design and development were carried out considering the technical and functional needs of the healthcare professionals thanks to the coordination between engineers and clinicians. In addition, comparing existing solutions [24, 99], IDE4ICDS has been developed with the ability of tolerating, controlling, and supporting deviations and inconsistencies of the real-world behaviors with respect proposal seeks to provide consistent and effective support to the human-centered system, still maintaining a high degree of flexibility and adaptability to the evolving needs, preferences, an expertise of the human agents in the health field.
Moreover, it is relevant to mention that our proposal has been validated from a clinical and software engineering perspective. On the one hand, IDE4ICDS was validated by implementing the CPGs for Type 2 Diabetes Mellitus (T2DM) in a real clinical environment at two health clinics in the Andalusian Public Health System in the southern Spanish region of Andalusia, involving 89 patients and obtaining a concordance analysis based on the kappa value. On the other hand, IDE4ICDS also showed good results and benefits from a software engineering perspective after comparing the software design and development process of the CPG for T2DM with a previous scenario, where a similar CPG was designed and developed without using the IDE4ICDS framework.
Finally, following this introduction, this paper is organized as follows. Section 2 presents some related works and computer-based execution CPG tools have been published in the scientific literature to know strengths and weaknesses. Section 3 describes the methodology used to design our proposal. Later, Section 4 and Section 5 details the theoretical results obtained and the supporting tool, respectively. Section 6 describes the validation case from a clinical and software engineering perspective. Finally, Section 7 presents the main conclusions drawn from this study and sets out some strategic considerations regarding future lines of research.

2 Related Works

This section presents related works that provide a background knowledge to carry out our proposal. Specifically, first, continuous process improvement lifecycles (in general terms and applied to the healthcare domain) are analyzed. Subsequently, the most recent model-driven proposals to address the digitisation of CPGs are discussed in order to identify what are the main challenges and limitations of these existing proposals. Finally, the most representative computer-based execution CPG tools are also studied to know their strengths and weaknesses.

2.1 Continuous Improvement Lifecycles

Over the last decades, methods and techniques have emerged for implementing continuous improvement processes to detect, correct or eliminate limitations or bottlenecks in operations management and to create business processes. Examples include TOC (Theory Of Constraints) [22], Six Sigma [88], Lean [75], and Lean Sigma [77]. These continuous improvement methods have traditionally been implemented in industrial [38], IT services [16], and manufacturing [4] contexts. In fact, most of the models mentioned originated in industrial environments. The Lean method, for example, was proposed by the Toyota Motor Company [6].
Some authors have proposed applying these methods in healthcare organizations in order to improve quality and efficiency while controlling costs [15, 57, 105, 107]. However, although the application of Lean strategies in a healthcare context sounds promising, some recent studies have shown that it is not possible to identify positive and significant impacts: firstly because most theoretical proposals are focused on identifying barriers and challenges in service-oriented processes for patients [30], and secondly, because significant differences between manufacturing and healthcare environments are evident when Lean’s principles are applied in healthcare (e.g., indetermination regarding which interested parties—patients, healthcare professionals, or healthcare organizations—will benefit from process improvement within the value chain of a healthcare service; limited ability to influence demand for the service; etc.) [90].
As mentioned above, continuous improvement methods (such as Lean) in health contexts have focused on clinical services and not on the comprehensive management of clinical guidelines.

2.2 Model-Driven Proposals to Digitize CPGs

The digitization of CPGs holds immense promise for improving patient care, clinician efficiency, and healthcare cost reduction. Model-driven approaches offer a compelling solution to the challenges that currently hinder this transition. However, despite the promising nature of this research area, there are hurdles to overcome before widespread adoption becomes a reality. While model-driven CPGs offer a compelling vision, substantial work is still needed to translate this vision into a globally accepted practice.
The research literature offers valuable insights into model-driven approaches for CPG development. For instance, Martinez et al. [64] provides a comprehensive overview of model-based approaches in this domain. This work highlights the potential benefits, existing challenges, and areas ripe for further exploration. However, a key limitation of this study is the lack of a concrete solution that can be readily adapted across different clinical scenarios. Another relevant study by Mathe et al. [65] presents a model-based approach for developing CDSS. While this approach demonstrates the utility of domain models for representing clinical knowledge and automatic CDSS generation, its primary focus lies on CDSS development rather than specifically addressing CPG digitization. Rahmani et al. [91] offer another example of a model-driven design approach. Their work presents a method for translating medical knowledge into verifiable finite state machine models, ensuring their correctness before code generation. This approach is showcased in the context of cardiac arrest guidance systems. However, the use of domain-specific languages like finite state machines limits the approach’s scalability for more general clinical settings.
In conclusion, our review of existing literature revealed a lack of a single, comprehensive approach that addresses all challenges associated with model-driven CPGs. Our proposed approach specifically targets these gaps by prioritizing the development of flexible and expressive models. This ensures the model can capture the complexities of clinical decision-making while remaining adaptable to diverse clinical scenarios. Furthermore, we advocate for a generalizable solution, aiming to create a framework applicable across various clinical areas. Finally, our approach emphasizes a user-centered design philosophy, ensuring clinician needs and workflows are central to the development process. This fosters user adoption and maximizes the impact of model-driven CPGs in real-world practice.

2.3 Computer-Based Execution CPG Tools

In recent years, some proposals and computer-based execution CPG tools have been published in the scientific literature to improve personalized patient care, reduce variability in clinical practice and improve healthcare resources [56, 111, 113]. This section presents the most representative related works to know their strengths and weaknesses after considering comparative studies published in this area, such as [41, 59, 85].
On the one hand, the Guide Project proposed an architecture that drew together medical and organizational issues through the Separation of Concerns paradigm, defining a Guideline Management System as a building block for optimizing communication between patient data management (medical issues) and organizational support. This CPG management lifecycle is broadly supported by different components, including formalization, dissemination, local adaption, and revisions processes, and requires participation at different levels, from certified organizations at the highest level down to national, regional, and local healthcare organizations. This proposal provides mechanisms of integration with electronic health record (EHR) and mechanisms of modeling (it uses the NewGuide language [18, 20]) and execution of CPG. This tool covers complete cycles of CPGs, and an agile mechanism to add new CPG models. However, this proposal does not provide some functionalities: mechanisms for integration with patient devices, personalized recommendations for each patient, indicator-based monitoring and version control of CPG models.
On the other hand, Guideline Interchange Format version 3 (GLIF3) Guideline Execution Engine (GLEE) [112] is a clinical guideline execution system that can interpret and execute guidelines encoded in GLIF3 [12], a representation format for clinical guidelines. GLEE can connect to a hospital’s Hospital Information System to access patient data, making it possible to implement clinical guidelines properly [110]. It can also integrate with clinical event monitors. GLEE-compatible guidelines can be created using modeling tools like GLIF Editor or Protégé-2000 [12, 110]. During guideline execution, healthcare providers receive recommendations based on scientific evidence for treating an illness, along with relevant information about the patient’s condition. GLEE also records the actual guideline execution, allowing for the review and analysis of the generated data. However, this proposal does not provide mechanisms for indicator-based monitoring of CPGs, nor version control of CPG models, nor a full management of the CPG lifecycle.
PROforma [36, 102] is a language in the logic programming and object-oriented modeling which is expressive enough to model guidelines considering artifacts as plans, actions, enquiries, decisions and data acquisition tasks. One particular property of the decision tasks in PROforma is their arguments, which represent rules in favor of and against taking certain possible decisions. These artifacts allow modeling complex tasks such as those carried out by healthcare professionals. Technical details can be found in [36, 102]. In addition, ProForma has integrated with electronic medical records and with medical vocabularies [1, 35]. This proposal also provides utilities to execute its models and define clinical recommendations. In this sense, some authors have extended PROforma with utilities to support recommendations and cognitive tasks (such as risk situation assessment, decision-making, and workflow management). For example, the CREDO proposal [35] is based on PROforma and defines a framework to improve decision-making in clinical practice from a cognitive approach. Finally, it has not been possible to locate evidence on monitoring mechanisms or version control of CPG models.
Moreover, ArdenSuite [69] is a software platform designed to provide CDSS solutions. It utilizes the Arden Syntax [26, 80], a widely accepted language for representing medical knowledge, to organize clinical information into modules known as Medical Logic Modules (MLMs). Each MLM encapsulates sufficient knowledge to make a clinical decision [95]. The platform encompasses two essential components [67]: (i) ArdenSuite IDE [66, 101], which IDE based on Eclipse IDE facilitating the creation and compilation of MLMs, as well as enabling visual modeling of clinical processes through plugins; (ii) ArdenSuite Server, which is the server-side component that executes compiled MLMs and manages their interactions with other systems, offering a comprehensive API for interoperability [66, 68]. ArdenSuite provides various demonstrations to showcase its features and assist users in familiarizing themselves with the platform. Overall, ArdenSuite serves as toolsets for developing and deploying CDS solutions (based on the Arden Syntax) and making informed clinical judgments. However, the integration with patient devices is not contemplated and it also lacks notification mechanisms associated with indicators. This tool also does not contemplate complete CPG cycles, it establishes a system of different modules. Finally, it lacks mechanisms to contain different versions of the CPGs included in the tool and also has no support tools.
In this sense, Anand et al. [2] proposes a pediatric tool based on the Arden Syntax proposal, which allows scalability and integration with EHRs. In addition, it provides mechanisms to model and execute clinical guidelines, as well as define personalized recommendations. However, this proposal does not provide monitoring mechanisms, nor notifications related to indicators, nor version control of CPG models, nor integration mechanisms with patient devices.
The Asgaard project is focused on temporal aspects of CPGs [97]. This proposal has been used to develop prototype applications for the management of diabetes, jaundice, and breast cancer [114]. It has mechanisms for modeling, execution and monitoring of CPGs, defining personalized recommendations. The Asgaard proposal was extended years later in the MobiGuide EU project [86, 87]. This project is based on the Asgaard project and the Asbru guideline-representation language developed within that project, which has been evaluated successfully clinically in Spain and in Italy during 2011 to 2015, in the domains of Gestational Diabetes (Barcelona) and Atrial Fibrillation (Pavia) [86, 87]. MobiGuide and Asgaard also include alarm mechanisms, which are sent to the patients and quality assessment of the clinical process. MobiGuide also proved the feasibility of representing multiple types of very different guidelines and running them on the very same Asbru-based engine. Other recent technical and clinical evaluations of the Asgaard/Asbru engines include the evaluation of the Asbru-based guideline-application engine in the domain of pre-eclampsia[98].
Guideline Acquisition, Representation and Execution (GLARE) [3, 104] is a clinical guideline management system developed in collaboration with one of Italy’s largest hospitals. It consists of two main modules: (i) an acquisition module for capturing and formalizing medical knowledge, which provides a graphical environment for creating and validating visual representations of clinical guidelines [103]; and (ii) a clinical guideline execution module for applying this knowledge to patient care using clinical decision support features, which assist healthcare professionals in making informed decisions about the best treatment or diagnostic options for their patients [104]. This platform has the same features as Asgaard proposal, but this tool also enables to represent temporal constrains of guidelines and actions, but the main advantages of the GLARE project is that the formal representation supports formal validation and verification of the guideline [11].
SAGE (framework for encoding and disseminating electronic clinical guidelines) [32, 106] proposes a guideline description model, a guideline authoring model and an execution environment. Its aim is to establish an infrastructure that allows guidelines to be shared in heterogeneous clinical information systems, in order to close the gap between theory and real life implementations. However, this proposal does not provide monitoring mechanisms, nor notification mechanisms related to indicators, nor full support for the CPG lifecycle, nor functionalities to allow scalability or model version control, among others. The SAGE project has been also implemented and evaluated in the Palo Alto Veterans Administration Health Care Center, and then in other medical centers, in the domain of hypertension, as the ATHENA project [44].
BRMN4CP [13] proposes an extension of Business Process Model and Notation (BPMN) to design and execute clinical guidelines. This proposal provides integration mechanisms with EHRs, but not with patient devices. It also has mechanisms to model, execute and monitor CPGs but it does not have a notification system related to indicators. It supports complete cycles of CPGs but does not provide personalized recommendations. It is also scalable and allows CPG extensions in an agile way but it does not support different versions of these CPG models.
KnowWe [5] is a semantic wiki seamlessly integrated into d3web [25], a platform that empowers the creation of expert diagnostic systems and caters to various application domains, including medical and therapeutic diagnoses, technical failure diagnostics, and technical device monitoring. To effectively represent clinical guidelines on KnowWe, a graphical modeling language called DiaFlux has been developed, enabling guidelines to be visualized as flowcharts [51]. The KnowWe proposal also allows these models to be executed with its d3web-core engine, which allows managing the persistence of clinical data, execution of clinical rules based on decision trees and heuristics, among other features. However, this proposal does not provide CPG monitoring mechanisms, nor notifications related to indicators, nor version control of CPG models.
Finally, after carrying out this brief analysis of the state of the art, our proposal presents advances in some of the features considered. These advances could be considered objective contributions of our proposal from the technological and formal point of view. On the one hand, the IDE4ICDS proposal is based on the definition of a flexible and extensible theoretical metamodels (c.f., Section 4.2), which provides mechanisms to add new concepts and features to the CPG models in a simple and fast way. This model-based architecture has facilitated the inclusion of monitoring concepts (such as dashboard and Key Performance Indicators (KPI), among others), which have been supported in the technological platform. In addition, the IDE4ICDS platform contemplates notifications and alarms when an indicator is out of range. On the other hand, IDE4ICDS also allows the versioning of models at a representative level, but it also allows several versions to be executed at the same time thanks to its process engine (c.f., Section 5).

3 Methods

Having formulated the starting hypotheses in the previous section, these have been addressed by combining two strategies: the MDE paradigm and methodological scientific principles [54]. The theoretical foundations of both strategies are explained in the following subsections.

3.1 Theoretical Principles of the MDE Paradigm

MDE is a well-known paradigm focused on models and how they can be used for software development. In MDE, concept representation is not relevant or at least not so relevant. The fundamental considerations are the following: which elements are included in the development process, how they interrelate and how they evolve in the development process. In fact, particular attention should be paid to two important elements in the MDE environment: metamodels and transformation rules[49].
On the one hand, metamodels are elements that normalize information about concepts in a domain. A metamodel defines constructors and relationships with constraints, making it possible to design models with valid semantics. Metamodels also allow different specific syntaxes (such as the textual and graphical syntaxes shown in the motivational example) to be combined, using the same lexicon and manipulating the same semantic artifacts. In this regard, the metamodel represents concepts but not the way to express them.
On the other hand, the MDE paradigm proposes the use of systematic mechanisms to generate new concepts from others in the same domain. These mechanisms are called transformation rules. A transformation is a relationship between elements in a source metamodel and elements in a target metamodel. Executing transformations therefore helps build a group of elements in target models that conform to their metamodels, using the information from a set of elements in source models which must also agree with their metamodels. In this case, the point of agreement guarantees that the transformation can be performed. Transformation elements define a systematic process regardless of the tool or programming language used.
In this research paper, MDE was used as mechanism to increase automation in the development process of software systems supporting clinical guideline management because it is one of the most trained paradigms in software engineering and has given satisfactory results when applied to different real software engineering environments, including software design [28], software development [37], software processes [39], software testing [94], and even software quality assurance [40], among others. In fact, its use is so widespread that there are even standards, like MDA (Model-Driven Architecture) [71], which provide a framework for structuring software specifications in MDE.

3.2 Scientific Methodology

DSRM [84] was applied in our paper as rigorous scientific methodology to produce a solution capable of answering our hypotheses (cf., Section 1) because DSRM is a very widely accepted and increasingly important paradigm for research into information software systems [53]. This methodology defines four main stages:
(1)
Problem identification and motivation. This phase aims to identify the problem our proposal tries to solve. For this purpose, we describe and analyze the motivation and the underlying problem for digitizing CPG from different perspectives. Firstly, Section 1 briefly introduces the need to digitize clinical guidelines from a clinical perspective in order to reduce the variability of medical practice as, so far, these guidelines are usually provided in textual form and without computer support. In addition, Section 1 also identifies the challenge and motivation in detail for carrying out this digitisation from a clinicians and software engineering perspective. Subsequently, after identifying the motivation and the problem to be solved, we conducted a review of works related to our hypotheses to identify what are the main advantages and weaknesses of these existing proposals (cf., Section 2): that is to say, (i) other proposals addressing specific aspects of CPG management and continuous improvement lifecycles (in general terms and applied to the healthcare domain); (ii) the most recent model-driven proposals to address the digitisation of CPGs; and (iii) the most representative computer-based execution CPG tools.
(2)
Objective of the solution. As mentioned above, most of the factors affecting the widespread implementation of computer-supported CPGs are mainly related to the inherent ambiguity in their statements due to the fact that CPGs are usually expressed in text format. This situation has meant that such systems are often developed on an ad hoc manner in each clinical institution, because each one has its own medical committees that decide how clinical guidelines should be used and applied by its healthcare professionals. In addition, there are other challenges and limitations to digitizing CPGs from a software engineering perspective; aspects related to the software maintenance and evolution of CPGs, the design of human-centered technological solutions, the growing variability of clinical practice among professionals and even between hospitals, and mechanisms to guarantee the quality of the software design and development process, among others, are the main challenges.
In this context, this phase aims to investigate and propose a framework based on a continuous improvement software lifecycle that includes model-driven mechanisms to facilitate the digitisation of CPGs. For this purpose and considering the multidisciplinary context of this topic, we are going to discover which are the human-centered elements that enable collaboration between healthcare professionals and software engineers during the software design and development process of computer-supported CPGs.
(3)
Design and development. After defining our objective, the design and development of our proposal involved several steps, which are described in Sections 4 and 5 in detail. On the one hand, we have proposed a framework (based on continuous improvement lifecycle) to address the definition, design and software development of the CPGs. This framework aims to standardise the software development process of computer-supported CPGs and to guide software engineering teams on what stages they should complete during this process. On the other hand, we have also defined model-driven mechanisms within our framework that allow approaching design and development of computer-supported CPGs from a human-centered perspective as well. Specifically, these mechanisms include: (i) designing a novel CPG metamodel and its associated semantic constraints, together with the MDE-based theoretical bases of the IDE4ICDS framework; and (ii) developing the IDE4ICDS platform, the tool which would support the application of our metamodels and automate parts of the theoretical framework. It should be mentioned that our software solution is platform-independent, since it was developed using standards like Unified Modeling Language (UML) [10] and ISO/IEC TR:24744 [33], which include guidelines that help improve consistency and uniformity in the definition of process models.
(4)
Demonstration and evaluation. In this stage of the DSRM methodology, we have evaluated the performance of our IDE4ICDS framework from clinical and software engineering perspectives. On the one hand, we have demonstrated the feasibility of our model-driven and human-centered approach through the digitisation of the CPG for T2DM. For this purpose, firstly, national and international knowledge sources on T2DM were selected, reviewed and used to define and model the CPG for T2DM applying IDE4ICDS (cf., Section 6.1), and, later, it was validated with 89 patients at two health clinics in the Andalusian Public Health System in the southern Spanish region of Andalusia [83]. On the other hand, we have also demonstrated promising results and benefits from a software engineering perspective (cf., Section 6.2) after comparing the software design and development process of the CPG for T2DM with a previous scenario, where a similar CPG was designed and developed without using the IDE4ICDS framework. For this purpose, effort and efficiency metrics were considered to assess the performance of our proposal in both scenarios.

4 IDE4ICDS as a Framework

4.1 Improved CPG (Digital) Implementation and Lifecycle Management

As described in Section 2.1, continuous improvement lifecycle in health contexts have focused on clinical services and not on the comprehensive management of clinical guidelines. Although there have been some studies which have proposed lifecycles for clinical guideline management, they have had shortcomings, mainly in aspects such as monitoring, continuous improvement, process orchestration, and so forth.
To fill these gaps, we propose a five-stage CPG continuous improvement lifecycle (c.f., Figure 1), which should be followed whenever a new digital CPG needs to be implemented in a healthcare system within our IDE4ICDS framework. Our lifecycle takes into account the related works and the knowledge held by different types of stakeholders: (i) healthcare professionals, who possess medical knowledge about CPGs, standards and clinical protocols; (ii) the healthcare organization, which manages and is familiar with the organizational aspects of healthcare activities; and (iii) software engineers, who have technical knowledge and modelling formalities about software systems. These stakeholders participate at different points in the lifecycle as explained below.
Fig. 1.
Fig. 1. IDE4ICDS: Continuous improvement lifecycle of CPG management on computer systems.
The first stage is (1) the modeling phase where the participation of healthcare professionals is required to formalize medical knowledge related to a CPG for a specific pathology. The participation of software engineers may be necessary as a modeling support team, but it is not mandatory because our proposal is supported by user-friendly software tools. The explicit medical knowledge (i.e., knowledge associated with published CPGs) and acquired medical knowledge (i.e., knowledge associated with skills and professional experience) of health professionals is digitally formalized, as explained in Section 5.1.
One advantage of this formalization is the possibility of evaluating and sharing knowledge among healthcare professionals to improve the efficiency and effectiveness of the protocol implementations and performance recommendations that are usually offered in CPGs. This evaluation also allows the clinical knowledge models themselves to be reformulated taking into account KPIs and knowledge acquired during clinical practice.
Once clinical knowledge has been digitally formalized in the previous stage, software engineers need to intervene to configure and deploy the CPG execution model, as well as obtain executable code to be integrated into the platform. This is the goal of the (2) configuration and deployment phase. The executable version needs an environment in which to run and which, in the context of IDE4ICDS, corresponds to software components such as process execution and decision rule execution engines (c.f., Section 5.1, IDE4ICDS architecture).
After configuring and deploying the executable version of the formal models of each CPG, healthcare professionals may enter clinical data into the IDE4ICDS platform to control their patients’ pathologies. This is done in the (3) execution phase of the IDE4ICDS lifecycle. Once the CPG have been deployed and executed in their respective execution environments, it is time to evaluate their effectiveness. The (4) monitoring phase oversees the controlling and monitoring of the performance of each CPG. This monitoring provides a rough view of the overall productivity of each CPG and is based on the definition of the KPIs in the modeling phase.
The last stage defined in the IDE4ICDS lifecycle is the (5) continuous improvement phase. In this stage, the KPIs that are being monitored are observed and the performance of the CPG is analyzed to obtain conclusions. The conclusions help optimize the efficiency, effectiveness, and quality of each CPG under execution, and improve clinical recommendations, implying better patient care.

4.2 IDE4ICDS as a Model-Driven Approach

The theoretical architecture of the IDE4ICDS proposal (c.f., Figure 2) supports each stage of the continuous improvement lifecycle described in Section 4.1 using model-driven mechanisms: i.e., metamodels and transformation rules.
On the one hand, IDE4ICDS includes three extensible metamodels which define abstract concepts to define, execute, orchestrate, and monitor CPGs:
(1)
«CPG Definition Metamodel» (c.f., Section 4.2.1), which contains concepts for modelling clinical guidelines from the user’s perspective: that is, it defines concepts like clinical roles, healthcare processes, clinical rules, clinical recommendations, descriptions of KPIs, and required connections with external systems;
(2)
«CPG E & O Metamodel» (c.f., Section 4.2.2), which defines those concepts related to the Execution and Orchestration (E & O) of the healthcare process associated with the CPG;
(3)
«CPG Monitoring Metamodel» (c.f., Section 4.2.3), which focuses on defining concepts related to the monitoring of the healthcare process associated with the CPG.
On the other hand, IDE4ICDS also includes the definition of a transformation protocol (c.f., Section 4.2.4) to systematically obtain the executable version of a clinical guideline from its models (previously listed). This protocol was built in several stages (as shown in Figure 2) and formalized using the Query, View, Transformation (QVT) [42] and Meta-Object Facility (MOF) Model to Text Transformation Language (MOFM2T) [72] standards, which allow the formal definition of Model-to-Model (M2M) and Model-to-Text (M2T) transformation rules, respectively. The stages of the transformation protocol are briefly explained below.
Fig. 2.
Fig. 2. Model-driven theoretical architecture of the IDE4ICDS platform.
Firstly, once the «CPG Definition Model» is modeled in accordance with the «CPG Definition Metamodel», with the structured information necessary to fully document a CPG, IDE4ICDS defines M2M transformation rules to systematically obtain an execution and orchestration model («CPG E&O Model») from «CPG Definition Model». This execution model is defined to conform to the «CPG E&O Metamodel», and its purpose is to define execution parameters for the subsequent running of the healthcare process associated with the CPG. It is important to mention that this execution model is considered a basic model (as shown in Figure 2): that is to say, software engineers can carry out modifications (i.e., manual transformations) on this model to adjust it to the technological context of the healthcare organization where the CPG is going to be executed. After conducting this setting, the final «CPG E&O Model» is obtained.
Then, once execution context, rules, clinical recommendations, and workflows have been defined in the previous stage, it is necessary to obtain executable code to run the healthcare process in the process engine. To do this, the proposal defines M2T transformation rules to systematically obtain executable code from «CPG E&O Model». This executable code will be deployed in the technological architecture of IDE4ICDS (Section 5.1) to support the execution phase of the IDE4ICDS lifecycle.
Subsequently, IDE4ICDS also proposes the systematic obtention of the «CPG Monitoring Model» from «CPG Definition Model» (which must have been previously defined). The monitoring model is defined to conform to the «CPG Monitoring Metamodel», and its purpose is to specify the mechanisms for the extraction, transformation and loading of clinical data (mechanisms that are necessary to calculate each KPI described in the «CPG Monitoring Model»). In this case, IDE4ICDS also proposes obtaining the final monitoring model. This is done in two steps. First, a basic model is systematically obtained from the definition model (as previously stated). Software engineers can then adjust this basic model to generate the final model. This setting makes it possible to indicate necessary information that is dependent on the technological context of execution.
Finally, once the last model has been defined, IDE4ICDS includes M2T transformations to systematically obtain executable code from it. This software code will also deploy in the technological architecture of IDE4ICDS (Section 5.1) in order to support the monitoring phase of the IDE4ICDS lifecycle.

4.2.1 CPG Definition Metamodel.

The exhaustive, complete CPG definition is the first step needed for the automatic management of clinical guidelines on decision support systems. In this regard, and as mentioned above, IDE4ICDS proposes a metamodel to fully define all the features of a CPG (c.f., Figure 3).
Fig. 3.
Fig. 3. «CPG Definition Metamodel» of IDE4ICDS.
Before continuing, it is relevant to mention that this definition metamodel is complex and contains numerous concepts and relationships between each concept. It was therefore convenient and necessary to divide it into several parts to improve its representability5:
Central concept. It refers to the «CPG metaclass» in Figure 3. This metaclass contains a set of static attributes (such as title, version, and scope, among others), but its most relevant information is related to the numerous relationships with other concepts. These concepts and relationships were grouped into the following blocks or sections of the definition metamodel: contextual information; clinical rules and recommendations; and healthcare process.
Contextual information. It refers to clinical and scientific information which make it possible to document aspects like authorship information («Authorship» metaclass), description of the disease to be treated («Disease» metaclass), treatment strategies («Strategy» metaclass), and also scientific evidence («ScientistEvidence» metaclass) of what criteria are appropriate for measuring the effectiveness of the clinical protocol defined by the CPG itself for the treatment of a specific disease.
Other contextual information may also be included, such as documentary annexes («Appendix» metaclass), possible lines of improvement («FutureWork» metaclass) and bibliographic references («Reference» metaclass), but these could be considered secondary information. Although this information and its metaclasses do not affect the subsequent automation of the clinical guide, it was included in our metamodel to document a complete, exhaustive solution.
Clinical rules and recommendations. This part of the «CPG Definition Metamodel» groups together abstract concepts that are necessary to structurally model clinical decision rules associated with the clinical guideline. The main concept in this part is called the «ClinicalRule» metaclass (see Figure 3) and refers to a set of systematic recommendations which help professionals and patients make decisions about the most appropriate therapeutic options when addressing specific pathologies [78].
After considering medical biographical sources (c.f., Section 3.2), it was possible to establish a taxonomy of recommendations or clinical rules. This taxonomy consists of 5 kinds of rules: (1) the risk factor rule, which refers to clinical recommendations and rules associated with risk factors for the disease being treated with the clinical guideline; (2) the clinical prevention rule, which refers to clinical recommendations for preventing symptoms of the disease being treated with the clinical guideline; (3) the diagnostic rule, which refers to recommendations for improving the clinical diagnosis of the disease; (4) the treatment rule, which defines recommendations for improving the pharmacological or healthcare treatment of the disease; and (5) the monitoring rule, which makes recommendations for monitoring the patient. Figure 3 represents this taxonomy using UML inheritance mechanisms between the «ClinicalRule» metaclass and its specialized metaclasses: «RiskFactorRule», «PreventionRule», «DiagnosisRule», «TreatmentRule», and «MonitoringRule» metaclasses, respectively.
Moreover, the «ClinicalRule» metaclass is defined with static attributes and relationships with other metaclasses of the «CPG Definition Metamodel»:
Regarding the static attributes, these ones are: «description», which describes the recommended action to be carried out by healthcare professionals; «header», which gives the title of the clinical rule; and «mandatory», which shows whether the rule is displayed on the platform when its condition is true.
Regarding relationships, the «ClinicalRule» metaclass is associated with [1.*] «ClinicalPracticeGuideline» metaclass and evaluates EHRs (represented by the «ClinicalData» metaclass in Figure 3). These data are managed in the healthcare process associated with the clinical guidelines and are usually filled in when the healthcare professional completes an activity in the process (the concept of healthcare process and activity are explained later).
The «ClinicalData» metaclass is considered the heart of rules and recommendations because it is considered to act as unit blocks for building logical conditions. These logical clauses are represented by the «Clause» metaclass (Figure 3) and related to each other using logical operators to build the full rule.
The logical clauses in turn contain [1.*] logical operands of the comparison type. These logical operands are represented in our metamodel by the «LogicalOperand» metaclass. Binary logical expressions were considered (specifically, logical expressions based on AND and OR operators). In this regard, the «LogicalOperand» metaclass is associated with the «ClinicalData» metaclass through the two associations («operand1 and «operand2 roles).
Healthcare process. This concept is considered in this article as a set of activities aimed at improving the care, treatment, and monitoring of patients and their pathologies. When these activities are related to one another, they constitute a workflow or healthcare process. Figure 4 shows the IDE4ICDS «Healthcare Process Definition Metamodel».
Its main concept is represented by the «HealthcareProcess» metaclass in Figure 4. The process consists of an ordered set of [3.*] elements (represented by the «ProcessElement» metaclass), the purpose of which differs according to its semantics within the workflow. Specifically, two types of main process elements were considered: «ControlElement» metaclass and «Activity» metaclass.
On the one hand, the first one defines the process control elements, which make it possible to establish the process structure and its different paths from the start node to the end node. This metaclass also has different behaviors depending on the value of its «type» attribute. More specifically, a control element can behave as a start element, an end element, a conditional element, or a fork/join element (thus making it possible to start/finish parallel branches on the workflow), depending on its configuration.
On the other hand, the «Activity» metaclass represents an action that has to be executed to continue with the process. This metaclass was specialized in three types of activity, according to its purpose: (i) an «OrchestrationActivity» metaclass, representing an activity performed by a machine; (ii) a «ComplexActivity» metaclass, which allows a process to be included within another process; and (iii) a «HumanActivity» metaclass, representing an activity performed by a human actor («Stakeholder» metaclass).
Regarding the «Stakeholder» metaclass, some semantic constraints were defined to build well-defined models. For example, Figure 4 shows the use of the «XOR» invariant between the «isParticipant» metaclass and («isResponsible» metaclass associations to ensure that the same stakeholder cannot occupy both roles. It was also necessary to verify that the job position of the stakeholder (the person/entity responsible for an activity) matches the minimum job position configured in the activity. This constraint is defined in Algorithm 1 using Object Constraint Language (OCL).
Performance information. As mentioned above, concepts related to measuring the performance of the CPG have also been included in our proposal. The top of Figure 4 shows these concepts.
The main concept is represented by «KeyPerformanceIndicator» metaclass, which allows the evaluation of the measurement objective based on defined analysis rules and suggests actions based on decision criteria. These actions are related to the required information («InformationNeeds» metaclass) to track a goal or constraint.
Moreover, «AbstractMeasure» metaclass has been defined as an abstract metaclass for grouping common properties to any measurement concept that can be defined later. This metaclass also allows to generalize the evaluation method of each KPI using the «Algorithm» metaclass, which defines how the measurement value is obtained and calculated.
Fig. 4.
Fig. 4. «Healthcare Process Definition Metamodel» of IDE4ICDS.

4.2.2 CPG Execution and Orchestration Metamodel.

After instantiating the «CPG Definition Metamodel», it is necessary to define and formally represent its structure and execution context. This representation allows us to specify how the CPG models (more specifically, the healthcare process model associated with the CPG) will be executed.
Figure 5 shows the «CPG EO Metamodel», which offers a complete and abstract specification to define the execution behavior of the process model within any process execution engine, as well as provide a description of the semantics of such execution. The «CPG EO Metamodel» describes static and dynamic semantics of the process execution models. For this purpose, we have considered following the Object Management Group (OMG’s) recommendations for executable UML models [81]. Static semantic refers to static information (i.e., properties or attributes) of each concept defined in «CPG EO Metamodel» and semantic constraints to ensure building well-formed models, whereas dynamic semantics refer to information managed at runtime and how each element of the metamodel is instantiated. Detailed information on the theoretical foundations of this metamodel, its elements and its semantics are briefly described below.
Fig. 5.
Fig. 5. «CPG EO Metamodel» of IDE4ICDS.
The main concept of this metamodel is represented by «ExecutionNodeClass» metaclass, which represents the executable version of any element in the process. These concepts were interrelated to build the process execution flow using the «ExecutionFlow» metaclass.
Since it was not possible to use UML to define the full semantics of the «ExecutionNodeClass» metaclass, we defined semantic invariants using OCL. Algorithm 2, for example, ensures that: (1) a process is not self-referenced to avoid indefinite recursion within the execution model; and (2) there is only one element of the process established as the starting point of the process.
The «ExecutionNodeClass» metaclass has static features (such as «name» and «description») and dynamic features (such as «status»). This last feature is essential because it constitutes the heart of the state machine that controls the internal lifecycle of each executable element.
Figure 6 shows this state machine whose transitions are triggered by operations defined in the «ExecutionFlow» metaclass. These operations are stereotyped as: (i) «CONST» (CONSTructor), which defines the constructor to create a new instance of the «ExecutionNodeClass» metaclass; or (ii) «SOP» (Status OPeration), which identifies operations to update or query in the internal status of the executable element.
Each operation was formally defined using an object-oriented language proposed by OMG for defining executable semantics [81]. Due to space constraints, it is beyond the scope of this paper to explain all the operations, but Algorithm 3, for example, shows the code associated with the «executeExecutableNode» operation, which launches the execution of an executable element.
Fig. 6.
Fig. 6. State machine associated to «ExecutionNodeClass» metaclass.

4.2.3 CPG Monitoring Metamodel.

This metamodel is closely related to the «CPG EO Metamodel». «CPG Monitoring Metamodel» (c.f., Figure 7) uses the measurement specifications—defined in the «Healthcare Process Definition Metamodel»—to identify the necessary measurement concepts that achieve the measurement goals («MeasurementGoal» metaclass) during the process execution.
The main concept of this metamodel is represented by «Dashboard» metaclass, which represents the panel which group the measurement goals and its related data. This concept is responsible for organizing and visualizing the measurement goals to be analyzed.
The previously mentioned measurement goals are monitored by [1.*] indicators (represented by «KeyPerformanceIndicator» metaclass in Figure 7), which measure the performance of running activities in the CPG. These running activities refer to the executable elements of the healthcare process, which are represented by «ExecutionNodeClass» metaclass in the «CPG EO Metamodel» (c.f., Section 4.2.2).
Fig. 7.
Fig. 7. «CPG Monitoring Metamodel» of IDE4ICDS.
Finally, our proposal also includes a concept to establish the visual configuration of each indicator that is shown in the dashboard. This configuration is represented by «Setting» metaclass and allows to set properties such as name, title or description of the chart, as well as its typology (area chart, bar chart or spline chart, among others).

4.2.4 The IDE4ICDS Model-Driven Transformation Protocol.

Section 4.2 explains the theoretical architecture of the IDE4ICDS proposal and how this architecture is based on on metamodels and transformation rules. This section aims to introduce this transformation rules. However, explanation of all transformation rules is out of the scope of this paper because it would become too extensive, but a couple of these rules are in detail described below.
On the one hand, Algorithm 4 shows the specification of the M2M transformation rule to obtain «HumanActivity» metaclass (defined in the «CPG EO Metamodel») from «HumanActivity» metaclass (defined in the «Healthcare Process Definition Metamodel»). This M2M rule transformation is named «toHumanExectClass» and maps both metaclasses using QVT notation.
This rule is responsible for initializing the attributes of the «HumanActivity» metaclass (from line 1 to line 6 of Algorithm 4) and for creating its relationships with other metaclasses (from line 7 to line 11 of Algorithm 4) in accordance with the relationships established in the «CPG EO Metamodel» (Figure 5).
Algorithm 4 also uses two types of helper methods. The first one is the «resolveone» method, which is a QVT method designed to return the associated object if it has been previously generated after transformation. If this associated object has not been generated, the «resolveone» method executes its transformation and returns its result. The second types are helper mapping methods that have been coded in IDE4ICDS to solve certain problems. For example, the «createPreConditions» and «createPosConditions» methods aim to generate the preconditions and postconditions, respectively, that govern the execution of the «HumanActivity» metaclass; whereas the «isInittialActivity» method (Algorithm 5) aims to calculate if this activity is the first activity in the process.

5 IDE4ICDS Tool Support

Throughout of Section 4, theoretical aspects of IDE4ICDS have been presented. Its lifecycle has been introduced, and how the choice of it can be made more flexible thanks to bidirectional transformations and traceability management has been discussed. Moreover, it has been described how the model-driven paradigm is instantiated within the methodology, showing in detail some of its metamodels and transformations. However, IDE4ICDS could not be considered a suitable solution if we do not instantiate all these ideas in a practical context, supported by tools that allow mixed teams of clinicians and software engineers to work effectively. Therefore in the following sections we present the architecture of the IDE4ICDS tool and how it is used in a real project.

5.1 Technological Architecture

In order to support the use of IDE4ICDS through a tool, the first step is to consider our requirements. The first is that the tools needed to incorporate all aspects related to MDE. In other words, we had to be able to include metamodels, transformations and profiles in a simple and suitable way that would guarantee us the ability to easily evolve IDE4ICDS MDE foundation. However, if IDE4ICDS wants to be an usable tool for industry, complex and abstract concepts like metamodels, transformations, and profiles have to be easy enough to be rightly dealt for the development teams. Therefore, we had to look for a simple environment with a reduced learning curve for mixed teams of clinicians and software engineers, but powerful enough to support our MDE environment.
The choice of the way in which we should approach the development of a tool that would support IDE4ICDS was not trivial, since a range of possibilities opened up before us to support IDE4ICDS’ MDE fundamentals and its CDSS, ranging from generating a new tool from scratch to the use and extension of existing tools, either open-source or commercial tools. It was discarded the option of developing a completely new tool, so we carried out a comparative review of between possible base MDE-supported tools and process execution engines with rule engine.
On the one hand, with regard to MDE-supported tools, tools such as Modelio, MagicDraw, WebGME, or the Eclipse ecosystem (Eclipse Modeling Framework - Graphical Modeling Framework, Sirius, Epsilon, and Theia-Graphical Language Server Platform Server) were evaluated, and finally Enterprise Architect (EA) [100] was the base solution selected6 because it provides the ability to model with UML and other modeling languages, it offers the possibility to develop new UML-profiles (meaning to extend the default version of UML), it allows to develop custom and user-friendly integrable GUIs inside itself, and it provides views such as the traceability matrix as long as certain properties are included in the modeling elements.
On the other hand, with regard to rule-driven process execution engines, after considering the requirements of IDE4ICDS and the findings of some surveys such as [70], we decided to use the well-known technologies BonitaOS [8] and Drools [9], which are an open-source process execution engine and a forward and backward chaining inference based rule engine, respectively.
In this context, Figure 8 shows the technological architecture of the IDE4ICDS platform. This architecture was designed using well-known software engineering patterns for designing computer systems [82] (such as, separation of concerns [14] and component-based software engineering [52] patterns) with encapsulated functionalities independent of each other, and divided by functional modules (both at front-end and back-end layer).
Fig. 8.
Fig. 8. Technological architecture of IDE4ICDS platform.
On the one hand, Definition Module is part of the front-end layer and its purpose is to be used by health professionals mainly to model CPG. It has been developed as a plugin within EA, offering wizards and user-friendly GUIs (which were developed using C# and other.NET Microsoft technologies). We also integrate UML-profiles into this IDE4ICDS plugin to instantiate each metamodel (c.f. Section 5.2). Specifically, three UML-profiles was implemented with their associated graphic editors: (i) «Clinical Guideline Modeling Tool»; (ii) «Clinical Rule Modeling Tool»; and (iii) «Healthcare Process Modeling Tool». Section 5.3 describes in detail how these editors are used in practice.
On the other hand, Transformation Rule Module is also part of the front-end layer (specifically, within the aforementioned IDE4ICDS plugin for EA) and its purpose is to be used by software engineers to obtain the executable version of the CPG definition models. It has been developed as a C#-encoded library to automatically execute both M2M and M2T transformation rules between models defined in the previous module. The systematic application of these transformation rules guarantees traceability between models when any change is made by users. The traceability of models is intrinsically supported by EA, improving the consistency of the relationships between different model versions of the same CPG.
Once the executable version of the CPG has been generated, the software engineers must deploy it in the Execution and Orchestration Modules. First, software engineers have to integrate the data entry forms within the execution module, which is related to the IDE4ICDS platform’s web portal; this web portal is part of the front-end layer of the platform. These forms are coded in C# and have been automatically generated from the CPG definition models. Later, software engineers have to connect these forms with the back-end orchestration module with the process engine and rules engine (which are included). In terms of software architecture, this module comprises a process execution engine (supported by BonitaOS) and a clinical rule decision engine (supported by Drools) Both engines manage flow control data, clinical data, and decision data on a clinical knowledge database, which are stored in two databases (knowledge database and clinical data database). These databases are directly managed by features included in the Data Persistence Module.
Finally, we also support the definition of KPIs when the CPG is modelled in the definition module to subsequently generate executable code using the transformation rules module. This executable code is structured as a library of methods (coded in C#) that perform the calculations associated with each KPI. After generating this code, the software engineer has to integrate this code within the Monitoring Module to measure and obtain indicators of the status of the CPG model execution.

5.2 UML-Profiles to Support the IDE4ICDS Metamodels

Following the recommendations of OMG, we have decided to design and develop UML-profiles to facilitate the instantiation of the IDE4ICDS metamodels because the UML extension provides a flexible and usable solution considering it is a well-known standard with many successful cases [29] and is supported by many modelling tools [93]. In short, UML provides three extension mechanisms: (i) «stereotype», defining the elements of a specific domain from which UML metaclasses would be extended; (ii) «tagged value», adding extra meta-attributes to any stereotyped element defined in the profile; and (iii) «constraint», identifying the conditions of the stereotyped elements to create well-defined models and extrapolate constraints (defined in the metamodel) to the profile.
The full description of our UML-Profile would be too long for this paper, so only a fragment of it is shown. In this sense, Table 1 shows the correlation between some IDE4ICDS stereotype and its related UML metaclass, as well as the justification of this relationship.
Table 1.
UML MetaclassIDE4ICDS StereotypeJustification
«Activity»«Process»It is a behavior specified as a sequence of subordinated units, using control and data flow model.
«ActivityNodes»«ProcessElement»They are used to model the individual steps in the behavior specified by an «Activity» of UML.
«ControlNodes»«ControlElement»They act as traffic switches managing the information flow across «ActivityEdges» (UML metaclass).
«ExecutableNodes»«Activity»They carry out the desired behavior of an «Activity» of UML.
Table 1. Some Relationships Between IDE4ICDS Stereotypes and UML Metaclass
Figure 9 also shows a fragment of the UML-Profile associated with the IDE4ICDS process modeling metamodel. It is important to note that two types of triangles are used in Figure 9: the dark triangle denotes the UML extension mechanism and the clear triangle represents the UML inheritance mechanism.
Fig. 9.
Fig. 9. Fragment of the UML-Profile associated with the IDE4ICDS metamodels.

5.3 Applying the Tool

This section describes how the IDE4ICDS platform is used in practical environments to support each phase of the lifecycle (c.f., Figure 1) considering the IDE4ICDS technology architecture (c.f., Figure 8).
On the one hand, during the modelling phase, healthcare professionals model and define the structure of the CPG using three graphical editors integrated in EA («Clinical Guideline Modeling Tool», «Clinical Rule Modeling Tool», «Healthcare Process Modeling Tool»), which are related to our CPG definition metamodel. Space restrictions make it impossible to show many images of this module in this article, but it is briefly illustrated in Figure 10. To facilitate its explanation, the figure has three labelled areas:
Fig. 10.
Fig. 10. Definition Module: IDE4ICDS plugin into EA.
Zone A. This shows the structure and organization of models from EA’s project browser. As can be seen in Figure 10, several packages are created automatically when the user creates a new project (i.e., a new CPG) in IDE4ICDS: «Repository of Indicators» stores metrics and indicators that have been defined to measure the execution and performance of the clinical guideline; «Repository of Rules» stores clinical rules; «Repository of Variables» stores clinical data used in the clinical guideline; and «Clinical Process» represents the healthcare workflow associated with the clinical guideline.
Area B. This shows the working space for users. As can be seen in Figure 10, models are built visually and graphically by dragging and dropping elements from the toolbox. The toolbox was obtained after implementing our UML-Profiles with EA. Figure 10 shows part of the workflow associated with a CPG.
Regarding our toolbox, readers may have doubts about the very small set of elements contained in the toolbox. This fact is justified and is related to Area C of Figure 10. Right from the beginning, the IDE4ICDS platform was designed to be user-centered and to reduce cognitive loads when in use. These decisions (especially, the reduction of the cognitive load) were taken to improve the acceptance and usability of IDE4ICDS in practical environments, as has been corroborated in some studies such as [45], which analyzed the cognitive overload of several languages (e.g., i* [74], BPMN [43], and UML [73]). These works have confirmed that complex modeling languages increase users’ cognitive overload and make the defined model more difficult to understand. This usability aspect has a major influence on users’ efficiency when using a language.
In this context, our toolbox therefore contains the minimum set of elements necessary to build the basic structure of the CPG, whereas the rest of the information is defined with user-friendly wizards and user-oriented GUIs.
Area C. This area displays one of the GUIs of the IDE4ICDS platform. Specifically, this GUI makes it possible to associate and link clinical rules to output links of a conditional element of the CPG process. When its condition is satisfied, this link allows clinical rules and recommendations to be executed. It is also possible to create new rules from this GUI (Figure 11) using the «Nueva Regla» («New Rule» in English) button. This button opens the wizard for defining new clinical rules. These forms also implement automatic semantic validations to ensure the building of well-formed models.
Fig. 11.
Fig. 11. Definition Module: Wizard for defining a new clinical rule into the IDE4ICDS plugin.
On the other hand, after defining the CPG definition model, the software engineer obtains its executable version (both for process engine, rule engine and indicator calculation) using the transformation rules module. This module is part of the IDE4ICDS plugin and its only user interface is within the menu EA (see top center area of Figure 10), which allows access to each transformation rule.
Once the executable version has been automatically obtained, it is deployed and configured in the execution and orchestration modules by the software engineer. Figure 12 briefly illustrates this module from the user’s perspective once the executable version of the clinical guideline is up and running. The screenshot at the back of this figure shows a form associated with one of the activities defined in the diabetes mellitus CPG. This form is executed in BonitaOS to collect clinical data from patients and send this information to the next activity when the healthcare professional finishes the current activity. This action launches the verification of some clinical rules with associated clinical recommendations for the healthcare professional.7 The action is captured by BonitaOS, which sends the clinical and context data to Drools to verify the rules associated with the current transition between activities. If the rule is verified, clinical recommendations are shown to the healthcare professional in the next activity (the blue text from the screenshot at the front of Figure 12).
Fig. 12.
Fig. 12. Execution and Orchestration modules integrated with BonitaOS and Drools (business process management suite and business rule management system, respectively).
Finally, regarding the monitoring and continuous improvement phase, the IDE4ICDS platform web portal provides different views to track the indicators that were defined during the modelling phase. In this sense, scorecards, timelines with the evolution of each indicator, and other features, are provided as shown in the Figure 13.
Fig. 13.
Fig. 13. Monitoring module of IDE4ICDS.

6 Evaluation

To assess the applicability of IDE4ICDS and its features we carried out validations from a clinical and software engineering perspective, that are presented in the Sections 6.1 and 6.2, respectively. This section does not aim to conduct a full experiment and it does not pretend to be exhaustive, since it is out of the scope of this paper, but introduce insights, issues and ideas for an evaluation with statistical significance. Our purpose is to present a multiple-validation study, gather initial data to validate our proposal, and obtain initial evidence to demonstrate the benefits of the approach.

6.1 Validation from a Clinical Perspective

This section briefly describes the methodological framework that we have used in our validation, as well as a summary of the results, since definitive results were published in an open-access paper including further details of the clinical validation [83].
First of all, ethical approval was obtained in the health organization based on the regional regulations (Andalusian Public Health System in the southern Spanish region of Andalusia) before involving it in the study execution. Likewise, informed consent procedures were defined, including informed consent and information sheets for the patients who were included in the study. Then, the IDE4ICDS platform was technically and functionally evaluated and validated through a proof-of-concept within a real application context in a Spanish public hospital. This proof-of-concept lasted ten months and was carried out in four stages:
(1)
Definition of the real application scenario. The Process of Comprehensive Care for T2DM [21] is the CPG selected to be deployed and executed on the platform. The proof-of-concept was evaluated with T2DM patients, who were recruited in two primary care centers8 belonging to the hospital area of the Virgen Macarena University Hospital in Seville, as part of the Andalusian Health Service (Spain).
However, a previous analysis stage was required to achieve this goal. Specifically, Spanish and international guidelines were analyzed and studied. At the international level, the CPG with the greatest impact in relation to T2DM is the Standards of Medical Care in Diabetes [31], which is updated annually by the American Diabetes Association. In Europe, one of the most important guidelines is the “QS6—Diabetes in adults quality standard” [79], maintained by the National Institute for Health and Care Excellence. In Spain, specifically in the Andalusian Public Health System, a periodically updated compendium of the reference guidelines indicated at international and European level has been developed and implemented with a more solid scientific basis. It is called the Integrated Care Process for diabetes mellitus [21], which defines the data and decision-making processes throughout the integrated care process.
(2)
Planning the proof-of-concept. This second stage aimed to define the inclusion criteria for the recruitment of patients, who were attended by health professionals using the IDE4ICDS platform. Specifically, the established criteria for the study were: patients over 18 years of age and diagnosed with T2DM. A total of 506 patients were identified as meeting the inclusion criteria, of whom 130 could be recruited (the remaining patients refused to participate), and 89 attended the clinical encounters where the follow-up could be carried out.
(3)
Execution of the proof-of-concept. The use and clinical validation of the IDE4ICDS platform was carried out through a prospective observational concordance study. First, each patient was attended by the healthcare professional in a conventional way (without using the platform) and according to the established clinical protocols. Later, each patient who agreed to participate in the study and signed the informed consent form was attended by a different healthcare professional using the IDE4ICDS platform to collect clinical information from the patient.
(4)
Evaluation of results. The evaluation of results was carried out through statistical analysis based on the Kappa-index statistical method [27], analyzing the concordance between the clinical recommendations automatically proposed by the IDE4ICDS platform, and the actions or recommendations carried out by healthcare professionals, in the case of the 89 involved patients. The Kappa-index method was selected because it represents the closest measure of the agreement to reality, taking into account the fraction of agreement due to chance, with the following strength of concordance: Poor: \(\lt\)0.20; Weak: 0.21–0.40; Moderate: 0.41–0.60; Good: 0.61–0.80; and Very good: 0.81–1.00.
After carrying out the statistical analysis referred to the concordance analysis, overall agreement between the recommendations provided by the IDE4ICDS platform and those recorded in each patient’s EHR was good (0.61–0.80). The total kappa index of the platform results as a 0.701, concluding a good agreement was reached. This result allowed us to perceive that the IDE4ICDS platform is reliable and has provided appropriate recommendations for each recruited patient for the defined application scenario [83].

6.2 Validation from a Software Engineering Perspective

The IDE4ICDS approach has shown good results after validation in real clinical environments for T2DM (as presented in the previous section), but we have also identified improvements and benefits from a software engineering perspective to value its suitability.
For this purpose, we compared the software design and development process of the CPG for T2DM with a previous scenario, where a similar CPG was designed and developed without using the IDE4ICDS proposal. Specifically, we refer to a CPG that addresses and supports the Intracytoplasmic Sperm Injection (ICSI) technique [60] in assisted reproductive treatment.9 Table 2 summarizes the context and scope of each CPG in terms of artifacts and software indicators. Both aspects are metrics that allow for an objective comparison of the size and complexity of each CPG. On the one hand, regarding the number of artifacts, the T2DM CPG is composed of 4 healthcare processes to analyze the patient risk factors, lifestyle parameters, diagnostic tests and patient treatments. These processes contain a total of 27 activities, 9 control elements and 21 clinical recommendations. In addition, 2 stakeholders (endocrinologists and nurses) are involved in these processes. Likewise, ICSI CPG is composed of 6 healthcare processes (associated with stages of hormonal stimulation, follicular aspiration, seminal preparation, among others), which contain 21 activities, 7 control elements and 17 clinical recommendations in total. In addition, 3 stakeholders (gynaecologist, laboratory technician, and nurses) are involved in the ICSI CPG. On the other hand, regarding software indicators, the implementation of the T2DM CPG included the development of 4 source-code files with 10,355 source-code lines in total (jointly considering manual-generated and automatic-generated source-code lines). Likewise, the implementation of the ICSI CPG included the development of 7 source-code files (mainly Structured query language and Python code), which included a total of 7,379 source-code lines.
Table 2.
  Artifacts of the CPGSoftware Indicators of the CPG
CPGIDE4ICDSTHPTATSTGTCRMG-SCFMG-SCLAG-SCFAG-SCLAutomation (%)
T2DMYes427292145,42744,92847.59
ICSINo621371777,379000
Table 2. Context of Our Case Studies
T2DM: CPG associated with the disease of Type 2 Diabetes Mellitus (IDE4ICDS-supported CPG), ICSI: CPG associated with the ICSI technique in assisted reproductive treatment (non-IDE4ICDS-supported CPG), THP: Total Healthcare Processes, TA: Total Activities, TS: Total Stakeholder, TG: Total Gateways, TCR: Total Clinical Recommendations, MG-SCF: Manual-generated source-code files, MG-SCL: Manual-generated source-code lines, AG-SCF: Automatic-generated source-code files, AG-SCL: Automatic-generated source-code lines.
After presenting the context of each case study, we describe software performance metrics that we have used to evaluate the benefits of our proposal when each CPG have been designed and developed (cf., Table 3). Specifically, we have measured: (i) the effort of each project member (measured in hours) to define requirements as well as design and develop a new CPG support system (using IDE4ICDS and without using it); and (ii) efficiency in terms of automation by generating new source-code systematically and automatically.
Table 3.
CPGIDE 4ICDSRoleDedication (%)Hours Per DayTotal (hours)Software Task
     R&DD&T
T2DM 4.5 monthsYesFANA604.8040532481
 DEV110086750675
 DEV210086750675
 CLI352.80236.25141.7594.50
    Total1,991.25465.751,525.50
ICSI 7.25 monthsNoFANA856.80924.38739.50184.88
 DEV110081,087.5001,087.50
 DEV210081,087.5001,087.50
 CLI554.40598.13358.88239.25
    Total3,697.501,098.382,599.13
Table 3. Comparison of Performance in Each Case Study
T2DM: CPG associated with the disease of Type 2 Diabetes Mellitus (IDE4ICDS-supported CPG), ICSI: CPG associated with the ICSI technique in assisted reproductive treatment (non-IDE4ICDS-supported CPG), FANA: Functional Analyst, DEV1: Developer, DEV2: Developer, CLI: Clinician, R & D: Requirements capture and Design tasks, D & T: Development and Testing tasks.
Regarding the efforts metric of the working team, it is interesting to mention that a similar team of software engineers and clinicians participated in each case study. In both cases, functional analyst (FANA), senior developer (DEV1), junior developer (DEV2), and clinical specialist (CLI) profiles were involved in both projects with the dedication shown in Table 3. The effort was measured in percentage of time spent and total hours required by each profile (with an average dedication of eight hours per day): (i) for the T2DM CPG, functional analyst, senior and junior developers, and clinicians dedicated 60% (in total 405 hours), 100% (in total 675 hours), 100% (in total 675 hours) y 35% (in total 236.25) of the time, respectively; (ii) whereas for the ICSI CPG, functional analyst, senior and junior developers, and clinicians dedicated 85% (in total 924.38 hours), 100% (in total 1,087.50 hours), 100% (in total 1,087.50 hours) and 55% (in total 598.13 hours) of the time, respectively. This dedication by profile is segregated in Table 3, considering the main tasks of the software development process: (i) requirements capture and design task, in which the profiles of functional analyst and clinicians are involved; and (ii) development and testing task, in which the profiles of developer (senior and junior) clinicians (in the functional testing stage) are involved.
After considering the measures of dedication and effort, it is possible to observe a significant reduction in the time spent in obtaining the software system to support each CPG (using and not using our model-driven approach): (i) regarding the requirements capture and design tasks, the work team spent 632.63 hours less using the IDE4ICDS proposal (which meant a reduction of 29.78% compared to the previous scenario in this task); (ii) regarding the development and testing tasks, the work team spent 1,073.63 hours less using the IDE4ICDS proposal (which represented a reduction of 36.99% of the time in this task). The reduction of effort has been obtained thanks to two main aspects: (i) source-code automation, which has improved the quality of software developments, allowing to reduce the duration of the functional testing task; and (ii) a better understanding of requirements definition and design by clinical profiles, who have been able to collaborate with functional analysts to define all the features of the clinical guidelines. This has been possible thanks to the human-centered design of our metamodels and specific languages for modelling clinical guidelines.
Finally, regarding the efficiency metric, it is important to mention that thanks to the IDE4ICDS proposal, it has been possible to automatically obtain the 47.59% associated with the source-code of the T2DM CPG (cf., Table 2). Specifically, the proposal automated the obtaining of 4,928 source-code lines (cf., Table 2). Obviously, the previous measures are only a case study, but the instanced processes have a complexity close to the average complexity of the processes implemented before using the proposal. Nevertheless, this validation offer very attractive results to continue with the improvement and refinement of the mechanisms driven by IDE4ICDS models.

7 Conclusions, Lessons Learned, and Future Works

CPGs formalize existing clinical knowledge in a specific context, but their representation is usually textual which causes ambiguity and variability when applying any CPG in clinical practice by healthcare professionals. In this article, we propose a model-driven and human-centered framework (IDE4ICDS) to mitigate this clinical situation and improve the software design and development of CDSS. In this respect, we propose the digitization of CPGs within a software lifecycle of continuous improvement (from their definition to their execution and monitoring) for reducing the variability of clinical practice when healthcare professionals monitor their patients’ pathology.
Furthermore, IDE4ICDS has been applied to validate the digitisation of the CPG for T2DM in two clinics of the Andalusian Public Health System in the southern region of Spain, involving 89 patients and obtaining a concordance analysis based on the kappa value. We have also analyzed the benefits of our model-driven approach from a software engineer’s perspective by comparing the results of the diabetes guide with a similar software project (without IDE4ICDS support). Both validations have returned valuable the following lessons learned.
On the one hand, the lessons learned by the clinicians were as follows: (i) reduction of the variability in the healthcare practice as the workflow is guided and suggests by the clinical recommendations of IDE4ICDS; (ii) possibility of providing feedback to improve the platform by recording the reasons why a recommendation is not followed in order to propose changes in the new versions of the clinical guideline; (iii) reduction of the patient follow-up time due to the automated support of the proposal; (iv) personalized attention to the different needs of each patient; (v) improvement of patient care; and (vi) reduction of iatrogenesis; and, as a consequence of all previous benefits, (vii) reducing healthcare costs; and (viii) increasing patient safety.
On the other hand, although we have also verified the use of model-driven paradigm facilitates the daily work of analysts and software experts (from a software engineering perspective in efficiency and effectiveness terms), concepts such as metamodels, UML profiles and transformations are very abstract and complex for them and non-technical users. However, this effect was mitigated with our proposal. Firstly, the collaboration between clinicians and software engineers was very fruitful to define the technical and functional requirements of the CPG for T2DM, thanks to the use of the domain-specific languages integrated in IDE4ICDS and the user-friendly and human-centered design of its user interfaces. These tools enabled a better understanding by healthcare professionals of the software design process for T2DM CPG, resulting in a more accurate and refined requirement model. As a consequence, it was possible to reduce the time spent during the software development stage thanks to the model-driven mechanisms incorporated in our proposal, which allowed to automate the generation of source-code (improving the quality of the software and requiring less time in testing and bug fixing stages).
Moreover, some future works will be discussed after presenting our proposal.
On the one hand, we plan to continue the validation and evaluation of our proposal on CPG for: (i) other pathologies (such as cardiac pathologies, oncological, autoimmune disease such as lupus, or even pluripathological patients, among others); and (ii) other important healthcare fields such as the nursing services to improve the preventive follow-up of patients with pathologies such as diabetic foot or patients with preeclampsia/eclampsia, among others. In this sense, our purpose is to ensure the generality and applicability of IDE4ICDS to different conditions and medical environments.
On the one hand, we plan to improve the functional and technical capabilities of IDE4ICDS by focusing on two technical aspects that have been highly requested by the healthcare professionals who validated our proposal. First, we will integrate our proposal with the healthcare IT ecosystems used in the Andalusian Health Service, developing and integrating an API based on the Fast Healthcare Interoperability Resources (FHIR)10 [109] (Fast Health Interoperability Resources) interoperability standard in IDE4ICDS. Later, once this FHIR-based API has been developed, we plan to extend our monitoring module to include early warnings on key clinical variables in the patient follow-up when the CPG is being executed into our proposal. We plan to disseminate these early warnings generated by IDE4ICDS to the rest of the healthcare IT system. Both improvements will represent an important technical advance because all healthcare professionals (of any medical specialty) will be able to know any clinical alteration of their patient in real time, which is especially relevant for multi-pathological patients who are treated by several healthcare professionals from different medical specialties.
Finally, the iterative nature of human-centered software design and development can often present a challenge [55] when software teams are faced with the necessary, rapid, product development lifecycles associated with healthcare IT software ecosystem. In fact, it has been observed that many otherwise excellent software products have failed in the marketplace due to poor design, while mediocre products have thrived due to attractive and user-friendly human-centered design [34]. Therefore, an important consideration in healthcare software design is the usability and human factors characteristics of the user interfaces and, hence, the experience they provide to the user. In this context, we plan to evaluate and validate the IDE4ICDS framework to ascertain the degree of acceptance, usability, ease of use, the training required to effectively utilize the platform, and its impact on clinical workflow efficiency when healthcare professionals use our proposal.

Acknowledgements

Thanks for her participation in the project to SERVIGIDE S. L. and Dr. María Asunción Martínez Brocca, Director of the Comprehensive Plan for Diabetes in Andalusia (Spain). Authors are grateful to Carlos III Health Institute for promoting the Network for Innovation in Medical Technologies and Health (“ITEMAS” in Spanish, CODE PT17/0005/0032).

Footnotes

1
Agency for Healthcare Research and Quality (USA). National Guideline Clearinghouse. Available at: https://www.ahrq.gov/gam/
2
Scottish National Health Service, Scottish Intercollegiate Guidelines Network. Available at: https://www.sign.ac.uk/
3
Cochrane Library. International non-for-profit organization Cochrane Collaboration. Available at: https://www.cochranelibrary.com/
4
Business process orchestration, or process orchestration, allows to connect workflows, technology, and people in end-to-end automated processes. At its core, process orchestration means automating and monitoring workflows to ensure all parts of your process are running smoothly.
5
Metaclasses that are colored dark gray represent specific parts of the overall metamodel
6
The features we identified in the review of MDE-supported tools are: UML metamodel or profile is included in the tool; ability of the tool to include new specific metamodels or UML profiles; how easy is to implement M2M or M2T (scripts, programming languages, and so forth); how easy is to deploy own developments in the tool; how easy the use of the tool is learned; how intuitive the tool is.
7
It is important to note that information related to rules, recommendations, activities, clinical data, and so forth, comes from formal models that have been previously defined by the healthcare professional using the definition module.
8
Primary care centers are the first level of access to the Andalusian public health system. Its objective is to provide comprehensive health care. It includes preventive, curative and rehabilitative care, as well as health promotion and health education.
9
ICSI [60] is an assisted reproduction treatment used during In Vitro Fertilization (IVF), where a single sperm is injected directly into the egg for the purpose of fertilization. During conventional IVF, the egg is placed in a culture dish with many sperm and fertilization occurs when one sperm penetrates the egg naturally.
10
The FHIR standard [109] is a set of rules and specifications for exchanging electronic health care data. It is designed to be flexible and adaptable, so that it can be used in a wide range of settings and with different health care information systems. The goal of FHIR is to enable the seamless and secure exchange of health care information, so that patients can receive the best possible care. The standard was created by the Health Level Seven International health-care standards organization.

References

[1]
Radhi R. Afandi, Abduljalil Radman, Mahadi Bahari, Lailatul Q. Zakaria, Muzaimi Mustapha, and Waidah Ismail. 2017. Ontology development in patients information system for stroke rehabilitation. In Proceedings of the 8th International Conference on Biomedical Ontology (ICBO ’17), 1–5.
[2]
Vibha Anand, Aaron E. Carroll, Paul G. Biondich, Tamara M. Dugan, and Stephen M. Downs. 2018. Pediatric decision support using adapted Arden Syntax. Artificial Intelligence in Medicine 92 (2018), 15–23.
[3]
Luca Anselma, Paolo Terenziani, Stefania Montani, and Alessio Bottrighi. 2006. Towards a comprehensive treatment of repetitions, periodicity and temporal constraints in clinical guidelines. Artificial Intelligence in Medicine 38, 2 (2006), 171–195.
[4]
Jiju Antony, Jorge L Escamilla, and Peter Caine. 2003. Lean sigma [production and supply chain management]. Manufacturing Engineer 82, 2 (2003), 40–42.
[5]
Joachim Baumeister, Jochen Reutelshoefer, and Frank Puppe. 2011. KnowWE: A semantic wiki for knowledge engineering. Applied Intelligence 35, 3 (2011), 323–344.
[6]
Ronald M. Becker. 1998. Lean manufacturing and the Toyota production system. Encyclopedia of World Biography (1998).
[7]
Rainer Blaser, M. Schnabel, C. Biber, M. Bäumlein, O. Heger, Mario Beyer, Egbert Opitz, Richard Lenz, and Klaus A. Kuhn. 2007. Improving pathway compliance and clinician performance by using information technology. International Journal of Medical Informatics 76, 2–3 (2007), 151–156.
[8]
Bonitasoft. 2024. Bonita Open Solution. (2024). Retrieved from https://www.bonitasoft.com/
[9]
Bonitasoft. 2024. Drools. (2024). Retrieved from https://www.drools.org/
[10]
Grady Booch. 2005. The Unified Modeling Language User Guide. Pearson Education India.
[11]
Alessio Bottrighi, Laura Giordano, Gianpaolo Molino, Stefania Montani, Paolo Terenziani, and Mauro Torchio. 2010. Adopting model checking techniques for clinical guidelines verification. Artificial Intelligence in Medicine 48, 1 (2010), 1–19.
[12]
Aziz A. Boxwala, Mor Peleg, Samson Tu, Omolola Ogunyemi, Qing T. Zeng, Dongwen Wang, Vimla L. Patel, Robert A. Greenes, and Edward H. Shortliffe. 2004. GLIF3: A representation format for sharable computer-interpretable clinical practice guidelines. Journal of Biomedical Informatics 37, 3 (2004), 147–161.
[13]
Richard Braun, Hannes Schlieter, Martin Burwitz, and Werner Esswein. 2014. BPMN4CP: Design and implementation of a BPMN extension for clinical pathways. In Proceedings of the IEEE International Conference on Bioinformatics and Biomedicine (BIBM ’14). IEEE, 9–16.
[14]
Johan Brichau, Maurice Glandrup, Siobhan Clarke, and Lodewijk Bergmans. 2001. Advanced separation of concerns. In Proceedings of the European Conference on Object-Oriented Programming. Springer, 107–130.
[15]
Nicola Burgess and Zoe Radnor. 2013. Evaluating Lean in healthcare. International Journal of Health Care Quality Assurance 26, 3 (2013), 220–235.
[16]
George Byrne, Dave Lubowe, and Amy Blitz. 2007. Using a Lean Six Sigma approach to drive innovation. Strategy & Leadership 35, 2 (2007), 5–10.
[17]
Michael D. Cabana, Cynthia S. Rand, Neil R. Powe, Albert W. Wu, Modena H. Wilson, Paul-Andre C. Abboud, and Haya R. Rubin. 1999. Why don’t physicians follow clinical practice guidelines?: A framework for improvement. Jama 282, 15 (1999), 1458–1465.
[18]
Paolo Ciccarese, Ezio Caffi, Lorenzo Boiocchi, Assaf Halevy, Silvana Quaglini, Anand Kumar, and Mario Stefanelli. 2003. The NewGuide Project: guidelines, information sharing and learning from exceptions. In Proceedings of the Conference on Artificial Intelligence in Medicine in Europe. Springer, 163–167.
[19]
Paolo Ciccarese, Ezio Caffi, Lorenzo Boiocchi, Silvana Quaglini, and Mario Stefanelli. 2004. A guideline management system. In Proceedings of the Medinfo. 28–32.
[20]
Paolo Ciccarese, Ezio Caffi, Silvana Quaglini, and Mario Stefanelli. 2005. Architectures and tools for innovative health information systems: The guide project. International Journal of Medical Informatics 74, 7–8 (2005), 553–562.
[21]
SAS Consejería Salud. 2018. Diabetes Mellitus - Proceso Asistencial Integrado. Junta de Andalucía, Sevilla 2010 (2018).
[22]
James Cox III and John Schleier. 2010. Theory of Constraints Handbook. McGraw-Hill.
[23]
John Crofton and DA Mitchison. 1948. Streptomycin resistance in pulmonary tuberculosis. British Medical Journal 2, 4588 (1948), 1009.
[24]
Gianpaolo Cugola, Elisabetta Di Nitto, Alfonso Fuggetta, and Carlo Ghezzi. 1996. A framework for formalizing inconsistencies and deviations in human-centered systems. ACM Transactions on Software Engineering and Methodology (TOSEM) 5, 3 (1996), 191–230.
[26]
Jeroen S. de Bruin, Klaus-Peter Adlassnig, Harald Leitich, and Andrea Rappelsberger. (2018). Separating business logic from medical knowledge in digital clinical workflows using business process model and notation and arden syntax. In Health Informatics Meets eHealth, IOS Press, 17–24.
[27]
Jeroen De Mast. 2007. Agreement and kappa-type indices. The American Statistician 61, 2 (2007), 148–153.
[28]
Francisco A. M. do Nascimento, Márcio F. S. Oliveira, and Flávio R. Wagner. 2007. ModES: Embedded systems design methodology and tools based on MDE. In Proceedings of the 4th International Workshop on Model-Based Methodologies for Pervasive and Embedded Software (MOMPES ’07). IEEE, 67–76.
[29]
Wojciech J. Dzidek, Erik Arisholm, and Lionel C. Briand. 2008. A realistic empirical evaluation of the costs and benefits of UML in software maintenance. IEEE Transactions on Software Engineering 34, 3 (2008), 407–432.
[30]
Antonio D’Andreamatteo, Luca Ianni, Federico Lega, and Massimo Sargiacomo. 2015. Lean in healthcare: A comprehensive review. Health Policy 119, 9 (2015), 1197–1209.
[31]
Nuha A. ElSayed, Grazia Aleppo, Vanita R. Aroda, Raveendhara R. Bannuru, Florence M. Brown, Dennis Bruemmer, Billy S. Collins, Kenneth Cusi, Sandeep R. Das, Christopher H. Gibbons, John M. Giurini, Marisa E. Hilliard, Diana Isaacs, Eric L. Johnson, Scott Kahan, Kamlesh Khunti, Mikhail Kosiborod, Jose Leon, Sarah K. Lyons, Lisa Murdock, Mary Lou Perry, Priya Prahalad, Richard E. Pratley, Jane Jeffrie Seley, Robert C. Stanton, Jennifer K. Sun, Crystal C. Woodward, and Deborah Young-Hyman. 2023. Introduction and methodology: Standards of care in diabetes—2023. Diabetes Care 46 (2023), 4 pages.
[32]
Prabhu Ram, David Berg, Samson Tu, Guy Mansfield, Qin Ye, Robert Abarbanel, and Nick Beard. 2004. Executing clinical practice guidelines using the SAGE execution engine. In Proceedings of the 11th World Congress on Medical Informatics (MEDINFO ’04). M. Fieschi (Ed.), Ios Pr Inc, 251.
[33]
ISO/IEC 2007. ISO/IEC TR24744:2007 Software and Systems Engineering Lifecycle Management Guidelines For Process Description. International Organization for Standardization, Geneva.
[34]
Elvis C. Foster and Elvis C. Foster. 2014. User interface design. Software Engineering: A Methodical Approach (2014), 187–205. DOI:
[35]
John Fox. 2017. Cognitive systems at the point of care: The CREDO program. Journal of Biomedical Informatics 68 (2017), 83–95.
[36]
John Fox, Nicky Johns, and Ali Rahmanzadeh. 1998. Disseminating medical knowledge: The PROforma approach. Artificial Intelligence in Medicine 14, 1–2 (1998), 157–182.
[37]
Robert France and Bernhard Rumpe. 2007. Model-driven development of complex software: A research roadmap. In Future of Software Engineering (FOSE ’07). IEEE, 37–54.
[38]
Matthew Franchetti, Kyle Bedal, Jenny Ulloa, and Selena Grodek. 2009. Lean and green: Industrial engineering methods are natural stepping stones to green engineering. Industrial Engineer 41, 9 (2009), 24–30.
[39]
J. A. Garcia-Garcia, José Gonzalez Enríquez, Laura Garcia-Borgoñon, Carlos Arévalo, and.E Morillo. 2017. A MDE-based framework to improve the process management: The EMPOWER project. In Proceedinfs of the IEEE 15th International Conference on Industrial Informatics (INDIN ’17). IEEE, 553–558.
[40]
J. A. García-García, J. Victorio, L. García-Borgoñón, M. A. Barcelona, F. J. Dominguez-Mayo, and M. J. Escalona. 2013. A formal demonstration of NDT-quality: A tool for measuring the quality using NDT methodology. In Proceedings of the 21st Annual Software Quality Management (SQM) Conference.
[41]
Julián Alberto García-García, Manuel Carrero, María José Escalona, and David Lizcano. 2023. Evaluation of clinical practice guideline-derived clinical decision support systems using a novel quality model. Journal of Biomedical Informatics (2023), 104573.
[42]
Tracy Gardner, Catherine Griffin, Jana Koehler, and Rainer Hauser. 2003. A review of OMG MOF 2.0 query/views/transformations submissions and recommendations towards the final standard. In Proceedings of the MetaModelling for MDA Workshop, Vol. 13. Citeseer, 41.
[43]
Nicolas Genon, Patrick Heymans, and Daniel Amyot. 2010. Analysing the cognitive effectiveness of the BPMN 2.0 visual notation. In Proceedings of the International Conference on Software Language Engineering. Springer, 377–396.
[44]
Mary K. Goldstein, Robert W. Coleman, Samson W. Tu, Ravi D. Shankar, Martin J. O’Connor, Mark A. Musen, Susana B. Martins, Philip W. Lavori, Michael G. Shlipak, Eugene Oddone, Aneel A. Advani, Parisa Gholami, and Brian B. Hoffman. 2004. Translating research into practice: organizational issues in implementing automated decision support for hypertension in three medical centers. Journal of the American Medical Informatics Association 11, 5 (2004), 368–376.
[45]
Nelson Goodman. 1968. Languages of ART: An Approach to a Theory of Symbols. The Bobbs-Merrill Company. Inc. New York, Indianapolis.
[46]
Evidence-Based Medicine Working Group. 1992. Evidence-based medicine. A new approach to teaching the practice of medicine. Jama 268, 17 (1992), 2420.
[47]
John Grundy, Hourieh Khalajzadeh, and Jenny McIntosh. 2020. Towards human-centric model-driven software engineering. In Proceedings of the International Conference on Evaluation of Novel Approaches to Software Engineering 2020. Scitepress, 299–238.
[48]
Michele Guerriero, Damian A. Tamburri, and Elisabetta Di Nitto. 2021. StreamGen: Model-driven development of distributed streaming applications. ACM Transactions on Software Engineering and Methodology (TOSEM) 30, 1 (2021), 1–30.
[49]
J. J. Gutiérrez, M. J. Escalona, and M. Mejías. 2015. A model-driven approach for functional test case generation. Journal of Systems and Software 109 (2015), 214–228.
[50]
Robin Harbour and Juliet Miller. 2001. A new system for grading recommendations in evidence based guidelines. Bmj 323, 7308 (2001), 334–336.
[51]
Reinhard Hatko, Joachim Baumeister, Volker Belli, and Frank Puppe. 2011. DiaFlux: A graphical language for computer-interpretable guidelines. In Proceedings of the International Workshop on Knowledge Representation for Health Care. Springer, 94–107.
[52]
George T. Heineman and William T. Councill. 2001. Component-based software engineering. Putting the Pieces Together, Addison-Westley 5, 1 (2001).
[53]
Alan Hevner and Samir Chatterjee. 2010. Design Research in Information Systems: Theory and Practice, Vol. 22. Springer, 978–1.
[54]
Alan R. Hevner, Salvatore T. March, Jinsoo Park, and Sudha Ram. 2004. Design science in information systems research. MIS Quarterly 28, (2004), 75–105.
[55]
Isaac Holeman and Dianna Kane. 2020. Human-centered design for global health equity. Information Technology For Development 26, 3 (2020), 477–505.
[56]
Natalia Iglesias, Jose M. Juarez, and Manuel Campos. 2020. Comprehensive analysis of rule formalisms to represent clinical guidelines: Selection criteria and case study on antibiotic clinical guidelines. Artificial Intelligence in Medicine 103 (2020), 101741.
[57]
Giovanni Improta, Giovanni Balato, Carlo Ricciardi, Mario A. Russo, Ida Santalucia, Maria Triassi, and Mario Cesarelli. 2019. Lean six sigma in healthcare: fast track surgery for patients undergoing prosthetic hip replacement surgery. The TQM Journal 31, 4, 526–540.
[58]
US Institute of Medicine. 2011. Clinical Practice Guidelines We Can Trust. Committee on Standards for Developing Trustworthy Clinical Practice Guidelines and Graham, Robin and Mancher, Michelle. National Academies Press, Washington, DC.
[59]
David Isern and Antonio Moreno. 2008. Computer-based execution of clinical guidelines: A review. International Journal of Medical Informatics 77, 12 (2008), 787–808.
[60]
Tarun Jain and Ruchi S. Gupta. 2007. Trends in the use of intracytoplasmic sperm injection in the United States. New England Journal of Medicine 357, 3 (2007), 251–257.
[61]
Matthias Kloppmann, Dieter Koenig, Frank Leymann, Gerhard Pfau, Alan Rickayzen, Claus von Riegen, Patrick Schmidt, and Ivana Trickovic. 2005. WS-BPEL extension for people–BPEL4people. Joint White Paper, IBM and SAP 183 (2005), 184.
[62]
Richard Lenz, Rainer Blaser, Mario Beyer, O. Heger, C. Biber, M. Bäumlein, and M. Schnabel. 2007. IT support for clinical pathways—Lessons learned. International Journal of Medical Informatics 76 (2007), S397–S402.
[63]
Marjolein Lugtenberg, Judith M. Zegers-van Schaick, Gert P. Westert, and Jako S. Burgers. 2009. Why don’t physicians adhere to guideline recommendations in practice? An analysis of barriers among Dutch general practitioners. Implementation Science 4, 1 (2009), 54.
[64]
Begoña Martínez-Salvador, Mar Marcos, Patricia Palau, and Eloy D. Mafé. 2023. A model-driven transformation approach for the modelling of processes in clinical practice guidelines. Artificial Intelligence in Medicine 137 (2023), 102495.
[65]
Janos L. Mathe, Jason B. Martin, Peter Miller, Akos Ledeczi, Liza M. Weavind, Andras Nadas, Anne Miller, David J. Maron, and Janos Sztipanovits. 2009. A model-integrated, guideline-driven, clinical decision-support system. IEEE Software 26, 4 (2009), 54–61.
[66]
Medexter Healthcare. 2018. Achieving System Connectivity Between Activiti BPMN Platform and ArdenSuite. Retrieved from https://www.medexter.com/ardensuite/ArdenSuite_Extension_for_Activiti_How_To.pdf
[67]
Medexter Healthcare. 2019. How to Compile, Test and Deploy MLMs with the ArdenSuite. Retrieved from https://www.medexter.com/ardensuite/How_To_Compile_Test_Deploy_MLMs.pdf
[68]
Medexter Healthcare. 2021. ArdenSuite Support. APIs. Retrieved from https://www.medexter.com/ardensuite-support/server/apis/
[69]
Medexter Healthcare. 2022f. Learning Center. Retrieved from https://www.medexter.com/products-and-services/learning-center
[70]
Ayman Meidan, Julián A. García-García, M. J. Escalona, and I. Ramos. 2017. A survey on business processes management suites. Computer Standards & Interfaces 51 (2017), 71–86.
[71]
Stephen J. Mellor, Kendall Scott, Axel Uhl, and Dirk Weise. 2004. MDA Distilled: Principles of Model-Driven Architecture. Addison-Wesley Professional.
[72]
OMG MOFM2T. 2008. OMG MOF Model to Text Transformation Language (OMG MOFM2T) Version 1.0. Object Management Group. Retrieved from http://www.omg.org/spec/MOFM2T/1.0
[73]
Daniel Moody and Jos van Hillegersberg. 2008. Evaluating the visual syntax of UML: An analysis of the cognitive effectiveness of the UML family of diagrams. In Proceedings of the International Conference on Software Language Engineering. Springer, 16–34.
[74]
Daniel Laurence Moody, Patrick Heymans, and Raimundas Matulevicius. 2009. Improving the effectiveness of visual representations in requirements engineering: An evaluation of i* visual syntax. In Proceedings of the 2009 17th IEEE International Requirements Engineering Conference. IEEE, 171–180.
[75]
Paul Myerson. 2012. Lean Supply Chain and Logistics Management. McGraw-Hill, New York, NY.
[76]
Ronnel Nallas and Jane Moon. 2016. Integration of automation and clinical decision support systems. In Proceedings of the Improving Health Management Through Clinical Decision Support Systems. IGI Global, 165–185.
[77]
Dag Näslund. 2008. Lean, six sigma and lean sigma: Fads or real process improvement methods? Business Process Management Journal 14, 3 (2008), 269–287.
[78]
Scottish Intercollegiate Guideline Network. 2008. SIGN 50: A Guideline Developers’ Handbook. Section 6: Forming Guideline Recommendations. Retrieved from https://www.sign.ac.uk/our-guidelines/sign-50-a-guideline-developers-handbook/
[80]
Tiago Oliveira, Paulo Novais, and José Neves. 2014. Development and implementation of clinical guidelines: An artificial intelligence perspective. Artificial Intelligence Review 42, 4 (2014), 999–1027.
[81]
OMG. 2010. Semantics of a Foundational Subset for Executable UML Models (fUML), Version 1.4. Object Management Group. Retrieved from https://doi.org/spec/FUML/1.4
[82]
Francis Palma, Hadi Farzin, Yann-Gaël Guéhéneuc, and Naouel Moha. 2012. Recommendation system for design patterns in software development: An dpr overview. In Proceedings of the 3rd International Workshop on Recommendation Systems for Software Engineering (RSSE ’12). IEEE, 1–5.
[83]
Carlos Luis Parra-Calderón, Esther Román-Villarán, Celia Alvarez-Romero, Germán A. Escobar-Rodríguez, Maria Asunción Martínez-Brocca, Alicia Martínez-García, Julián A. García-García, and María J. Escalona-Cuaresma. 2023. A prospective observational concordance study to evaluate computational model-driven clinical practice guidelines for Type 2 diabetes mellitus. International Journal of Medical Informatics 178 (2023), 105208.
[84]
Ken Peffers, Tuure Tuunanen, Marcus A. Rothenberger, and Samir Chatterjee. 2007. A design science research methodology for information systems research. Journal of Management Information Systems 24, 3 (2007), 45–77.
[85]
Mor Peleg. 2013. Computer-interpretable clinical guidelines: A methodological review. Journal of Biomedical Informatics 46, 4 (2013), 744–763.
[86]
Mor Peleg, Yuval Shahar, Silvana Quaglini, Tom Broens, Roxana Budasu, Nick Fung, Adi Fux, Gema García-Sáez, Ayelet Goldstein, Arturo González-Ferrer, Hermie Hermens, M Elena Hernando, Val Jones, Guy Klebanov, Denis Klimov, Daniel Knoppel, Nekane Larburu, Carlos Marcos, Iñaki Martínez-Sarriegui, Carlo Napolitano, Àngels Pallàs, Angel Palomares, Enea Parimbelli, Belén Pons, Mercedes Rigla, Lucia Sacchi, Erez Shalom, Pnina Soffer, and Boris van Schooten. 2017. Assessment of a personalized and distributed patient guidance system. International Journal of Medical Informatics 101 (2017), 108–130.
[87]
Mor Peleg, Yuval Shahar, Silvana Quaglini, Adi Fux, Gema García-Sáez, Ayelet Goldstein, M. Elena Hernando, Denis Klimov, Iñaki Martínez-Sarriegui, Carlo Napolitano, Enea Parimbelli, Mercedes Rigla, Lucia Sacchi, Erez Shalom and Pnina Soffer. 2017. MobiGuide: A personalized and patient-centric decision-support system and its evaluation in the atrial fibrillation and gestational diabetes domains. User Modeling and User-Adapted Interaction 27, 2 (2017), 159–213.
[88]
Low S. Pheng and Mok S. Hui. 2004. Implementing and applying Six Sigma in construction. Journal of Construction Engineering and Management 130, 4 (2004), 482–489.
[89]
Amir Qaseem, Vincenza Snow, Alice Gosfield, David Gregg, Keith Michl, David Wennberg, Kevin B. Weiss, and Eric C Schneider. 2010. Pay for performance through the lens of medical professionalism. Annals of Internal Medicine 152, 6 (2010), 366–369.
[90]
Zoe J Radnor, Matthias Holweg, and Justin Waring. 2012. Lean in healthcare: The unfilled promise? Social Science & Medicine 74, 3 (2012), 364–371.
[91]
Maryam Rahmaniheris, Yu Jiang, and Lui Sha. 2016. Model-driven design of clinical guidance systems. arXiv:1610.06895.
[92]
Frans Rutten, Werner Brouwer, and Louis Niessen. Practice guidelines based on clinical and economic evidence. Indispensable tools in future market oriented health care. The European Journal of Health Economics 6, 2 (June 2005) 91–93. DOI:. 15750827.
[93]
Safdar A. Safdar, Muhammad Z. Iqbal, and Muhammad U. Khan. 2015. Empirical evaluation of UML modeling tools–a controlled experiment. In Proceedings of the European Conference on Modelling Foundations and Applications. Springer, 33–44.
[94]
Alberto Salido, Julián A. G. García, José Ponce, and Javier J. Gutierrez. 2014. Tests management in CALIPSOneo: A MDE solution. Journal of Software Engineering and Applications 7, 06 (2014), 506.
[95]
Matthias Samwald, Karsten Fehre, Jeroen De Bruin, and Klaus-Peter Adlassnig. 2012. The Arden Syntax standard for clinical decision support: Experiences and directions. Journal of Biomedical Informatics 45, 4 (2012), 711–718.
[96]
Douglas C. Schmidt. 2006. Model-driven engineering. Computer-IEEE Computer Society 39, 2 (2006), 25.
[97]
Yuval Shahar, Silvia Miksch, and Peter Johnson. 1998. The Asgaard project: A task-specific framework for the application and critiquing of time-oriented clinical guidelines. Artificial Intelligence in Medicine 14, 1–2 (1998), 29–51.
[98]
Erez Shalom, Yuval Shahar, and Eitan Lunenfeld. 2016. An architecture for a continuous, user-driven, and data-driven application of clinical guidelines and its evaluation. Journal of Biomedical Informatics 59 (2016), 130–148.
[99]
Han-Chin Shing. 2021. A Human-Centric Approach to NLP in Healthcare Applications. Ph. D. Dissertation. University of Maryland, College Park.
[100]
Sparks. 2024. Enterprise Architect. (2024). Retrieved from https://sparxsystems.com/
[101]
Martin Spineth, Andrea Rappelsberger, and Klaus-Peter Adlassnig. 2018. Achieving interoperability between Arden-Syntax-based clinical decision support and openEHR-based data systems. Studies in Health Technology and Informatics 248 (2018), 338–344.
[102]
David R. Sutton and John Fox. 2003. The syntax and semantics of the PRO forma guideline modeling language. Journal of the American Medical Informatics Association 10, 5 (2003), 433–443.
[103]
Paolo Terenziani, Stefania Montani, Alessio Bottrighi, Mauro Torchio, Gianpaolo Molino, Luca Anselma, and Gianluca Correndo. 2003. Applying artificial intelligence to clinical guidelines: The GLARE approach. In Proceedings of the Advances in Artificial Intelligence (AI*IA ’03). Springer, 536–547.
[104]
Paolo Terenziani, Stefania Montani, Alessio Bottrighi, Mauro Torchio, Gianpaolo Molino, and Gianluca Correndo. 2004. The GLARE approach to clinical guidelines: Main features. In Computer-based Support for Clinical Guidelines and Protocols. IOS Press, 162–166.
[105]
John S. Toussaint and Leonard L. Berry. 2013. The promise of Lean in health care. In Mayo Clinic Proceedings, Vol. 88. Elsevier, 74–82.
[106]
Samson W. Tu, James R. Campbell, Julie Glasgow, Mark A. Nyman, Robert McClure, James McClay, Craig Parker, Karen M. Hrabak, David Berg, Tony Weida, James G. Mansfield, Mark A. Musen, and Robert M. Abarbanel. 2007. The SAGE Guideline Model: Achievements and overview. Journal of the American Medical Informatics Association 14, 5 (2007), 589–598.
[107]
Jaap Van Den Heuvel, Ronald J. M. M. Does, and John P. S. Verver. 2005. Six Sigma in healthcare: Lessons learned from a hospital. International Journal of Six Sigma and Competitive Advantage 1, 4 (2005), 380–388.
[108]
Jan P. Vandenbroucke. 1996. Evidence-based medicine and “médecine d’observation”. Journal of Clinical Epidemiology 49, 12 (1996), 1335–1338.
[109]
Amit Walinjkar and John Woods. 2018. FHIR tools for healthcare interoperability. Biomedical Journal of Scientific and Technical Research 9, 5 (2018).
[110]
Dongwen Wang, Mor Peleg, Samson W. Tu, Aziz A. Boxwala, Omolola Ogunyemi, Qing Zeng, Robert A. Greenes, Vimla L. Patel, and Edward H. Shortliffe. 2004. Design and implementation of the GLIF3 guideline execution engine. Journal of Biomedical Informatics 37, 5 (2004), 305–318.
[111]
Dongwen Wang, Mor Peleg, Samson W. Tu, Edward H. Shortliffe, and Robert A. Greenes. 2001. Representation of clinical practice guidelines for computer-based implementations. Medinfo 10, Pt 1 (2001), 285–9.
[112]
Dongwen Wang and Edward H. Shortliffe. 2002. GLEE–a model-driven execution system for computer-based implementation of clinical practice guidelines. In Proceedings of the AMIA Symposium. American Medical Informatics Association, 855–859.
[113]
Steven H. Woolf, Richard Grol, Allen Hutchinson, Martin Eccles, and Jeremy Grimshaw. 1999. Potential benefits, limitations, and harms of clinical guidelines. Bmj 318, 7182 (1999), 527–530.
[114]
Kai Zheng, Rema Padman, Michael P. Johnson, and Sharique Hasan. 2010. Guideline representation ontologies for evidence-based medicine practice. In Handbook of Research on Advances in Health Informatics and Electronic Healthcare Applications: Global Adoption and Impact of Information Communication Technologies. Khalil Khoumbati, Yogesh K. Dwivedi, Aradhana Srivastava, and Banita Lal (Eds.), IGI Global, 234–254.

Cited By

View all
  • (2024)A Question-and-Answer Scheme of Clinical Practice Guidelines Based on SDA*Proceedings of the 2024 5th International Symposium on Artificial Intelligence for Medicine Science10.1145/3706890.3706906(96-101)Online publication date: 13-Aug-2024

Index Terms

  1. IDE4ICDS: A Human-Centric and Model-Driven Proposal to Improve the Digitization of Clinical Practice Guideline

      Recommendations

      Comments

      Information & Contributors

      Information

      Published In

      cover image ACM Transactions on Software Engineering and Methodology
      ACM Transactions on Software Engineering and Methodology  Volume 33, Issue 7
      September 2024
      943 pages
      EISSN:1557-7392
      DOI:10.1145/3613705
      • Editor:
      • Mauro Pezze
      Issue’s Table of Contents
      This work is licensed under a Creative Commons Attribution International 4.0 License.

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Publication History

      Published: 27 September 2024
      Online AM: 28 June 2024
      Accepted: 04 June 2024
      Revised: 27 April 2024
      Received: 18 January 2024
      Published in TOSEM Volume 33, Issue 7

      Check for updates

      Author Tags

      1. Clinical practice guideline
      2. human-centric software design
      3. model-driven engineering paradigm
      4. computer-interpretable clinical guidelines
      5. knowledge representation

      Qualifiers

      • Research-article

      Funding Sources

      • IDE4ICDS
      • EQUAVEL

      Contributors

      Other Metrics

      Bibliometrics & Citations

      Bibliometrics

      Article Metrics

      • Downloads (Last 12 months)552
      • Downloads (Last 6 weeks)162
      Reflects downloads up to 15 Jan 2025

      Other Metrics

      Citations

      Cited By

      View all
      • (2024)A Question-and-Answer Scheme of Clinical Practice Guidelines Based on SDA*Proceedings of the 2024 5th International Symposium on Artificial Intelligence for Medicine Science10.1145/3706890.3706906(96-101)Online publication date: 13-Aug-2024

      View Options

      View options

      PDF

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader

      Login options

      Full Access

      Media

      Figures

      Other

      Tables

      Share

      Share

      Share this Publication link

      Share on social media