CMMN evaluation: the modelers’ perceptions of the main notation elements

Abstract

Case Management Model and Notation (CMMN) has been introduced as a graphical modeling language targeting the modeling of human-centric processes. Despite its growing reputation since 2016, when the OMG standant was released, the usage and the adoption potential of CMMN is not yet evaluated. The goal of this paper is to evaluate CMMN language and the contribution of its main notation elements to its future adoption, based on the experience of modelers. A CMMN workshop was conducted, where groups of modelers modeled two different human-centric, real-world processes with CMMN. The effectiveness and efficiency of the language and modelers’ usage experience were evaluated. Their perception of the role of the CMMN notation elements to their future adoption CMMN have been recorded through a survey. A multi-criteria decision making method (Analytic Hierarchy Process–AHP) was utilized for analyzing the answers and generating the results. The evaluation results showed that CMMN language could be adopted for modeling non-structural processes and the study participants showed a positive attitude towards adopting CMMN driven by the fact that they overall perceived it as useful. To the best of our knowledge, this is the first attempt to evaluate CMMN language’s usability and prospects of adoption. Moreover, this is the first empirical study that explores the syntax of a process modeling language and its effect on its usage and adoption. Overall, since interest in CMMN is increasing, this work could inspire future researchers and practitioners to further explore the CMMN usage and adoption potential.

Appendices

Appendix A: Cases description

1.1 A.1 Patient treatment

To begin with, the patient is admitted to a hospital’s Medicine Clinic if he/she needs to be hospitalized, a decision that is taken at the Emergency Department. The Emergency Department personnel provide the physicians of the clinic with information regarding the clinical status of the patient, such as medical history and any examinations that have been done or scheduled.

Based on this initial information, the physicians of the Medicine Clinic start the treatment of the patient. They specify a diagnosis for the patient and prescribe the medication accordingly. Such information is registered into the patient’s file. During treatment, a clinical examination takes place every morning by the physicians, aiming at monitoring the patient’s clinical course. To this end, laboratory and/or imaging examinations may be scheduled. The results, which are also registered into the patient’s file, are evaluated by the physicians and if necessary the diagnosis and medication are revised. There may be cases that the physicians will need to consult a specialist in order to conclude about the patient’s health problem or about the way the patient should be treated.

The nursing personnel aids in the treatment process through operations regarding, for example, the preparation and administration of the specified medication, blood drawing and measurements of vital signs. Medication administration and measurements are performed at the times specified by the physician. The measured values are written in the patient’s chart. Moreover, the nursing personnel keeps notes of anything remarkable regarding the patient, for example, a sign they observed, as well as of any action they performed by their own initiative, for example any ad-hoc medication they may have administered to the patient.

During treatment, several unexpected situations may arise, which may lead to ad hoc clinical, laboratory or imaging examinations, as well as to reconsideration of the medication administered or even of the diagnosis specified so far. The patient may need to be transferred to the Intensive Care Unit or to undergo an urgent surgery.

The need for a patient to remain hospitalized is daily examined after the morning clinical examination based on the data gathered up to that point. If it is decided that the patient does not need further hospitalization, the treatment process ends and the patient is discharged.

1.2 A.2 Product exchange

First of all, there are two types of users, i.e., guest and registered users that differentiate themselves in the permissions that they get granted regarding the use of the platform.

More specifically, when someone visits the web platform for the first time, he gets prompted to register, by creating a user account. This account can be created either by signing up via an email and a password or via a social network account. After a successful registration, the, from now on, registered platform user, is able to submit an advertisement donating or exchanging an item, to declare interest for an existing EE product and propose an offer to acquire it, as well as to communicate with any other user who owns a desirable electric device.

Moreover, a registered user is not only able to search a product based on some conditions, namely, filters like item categories, item state, donating-user region, but also to either suggest changes regarding the item’s category for which he/she is searching, or even to comment in an advertisement that he/she had expressed interest for. That way, the appropriate users will be notified for either the category change proposal or the commenting in an advertisement.

Finally, registered users have a profile in which they are able to be notified for any recycling actions taken via a news-feed as well as being informed for general topics regarding recycling and its benefits. Within each user’s profile, a calendar exists via which a user can be informed for any recycling events taking place.

Appendix B: Analytic hierarchy process

1.1 B.1 Methodology description

The AHP adopts a hierarchical form using three conceptual levels, as it is projected in Fig. 5. In the first level, the objective of the decision making process is defined. In the next level, the number of elements on which the evaluation will be based is identified. The various elements are denoted by $E_{k}$, where k is an integer with $1 \le k \le N$, and N is the total number of elements. In this paper we consider the CMMN elements. In the last level of the hierarchy we have the factors, denoted by $F_{i}$, where i is an integer with $1 \le i \le J$ and J the total number of factors. [12].

According to the AHP, in order to explore which of the CMMN elements contributes most towards the intention to adopt CMMN, then one must perform pairwise comparisons and evaluate the weights of importance of each CMMN element. In the same context, we have to explore the scores of each factor $F_i$ for serving $E_k$ Pairwise comparisons is a fundamental part of the AHP, according to which the participants compare the elements/factors in pairs instead of assigning their weights in a single step. This reduces the influence of subjective points of views, associated with eliciting the weights directly.

Each participant m ($1 \le m \le M$, where M is the group of participants), compares all possible combinations of CMMN elements by filling out the N x N pairwise comparison matrix $\mathbf{P} ^{( m )}$=$[P_{ij}^{(m)}]$, the elements of which signify the importance of a CMMN element $E_{i}$ compared to another CMMN element $E_{j}$ towards Intention to Adopt CMMN, assigning values from the nine-level scale[12]. The participants need to complete only the upper triangular elements since PWC is a reciprocal matrix. The weights $w_{k}^{(m)}$ of a CMMN element $E_{k}$ according to participant m are calculated by solving the eigenvalue problem, according to which the eigenvalues of $\mathbf{P} ^{( m )}$ are calculated and the eigenvector $\mathbf{x} _1^{( m )}$ = $[ x_{1k}^{( m )} ]$ associated with the largest eigenvalue $\lambda _{max}^{( m )}$ is determined [52]. The weight $w_k^{( m )}$ are obtained normalizing the sum of the eigenvectors $\mathbf{x} _1^{( m )}$ of the matrix to unity,

$$\begin{aligned} w_k^{(m)}=x_{1k}^{(m)}\left[ \sum _{l=1}^{N}x_{1l}^{(m)}\right] ^{-1} \end{aligned}$$

(1)

After all the comparisons have been completed, the average weight $w_{k}$ for each element $E_{k}$ is calculated by averaging out the weights $w_{k}^{(m)}$ obtained by the M participants.

$$\begin{aligned} w_{k}=1/M\sum _{m=1}^{M}w_{k}^{(m)} \end{aligned}$$

(2)

Care should be taken so that the pairwise comparison matrices produced by the participants are as consistent as possible in terms of proportionality and transitivity [8]. The PWC matrix $\mathbf{P} ^{(m)}$ is said to be perfectly consistent if all its elements are of the form $P_{ij}^{(m)} = q_i^{(m)}/q_j^{(m)}$, where $q_i^{(m)},q_j^{m}$ are positive real numbers.The consistency ratio (C.R.) is one measure for consistency can be readily obtained from the pairwise comparison matrices as described in [8]. In our case, the C.R. values were less than 0.1 (ranged between 0.015 to 0.076) which is considered acceptable [39].

The same procedure is followed for the factors of the second level of hierarchy. Towards this end, the factors are pairwise compared with respect to each element and for each factor $F_{i}$ one obtains the relative scores $S_{ik}$ under element $E_{k}$, depicting the score of $F_i$ for serving $E_k$. Finally, one can evaluate to what extent each acceptance factor contribute to the Intention to Adopt CMMN, by multiplying the relative scores $S_{ik}$ by the weight $w_{k}$ of the corresponding element and estimate the overall weight $R_i$.

$$\begin{aligned} R_{i}=\sum _{k=1}^{N}S_{ik}w_{k} \end{aligned}$$

(3)

1.2 B.2 AHP results verification

Changes of the group of participants. The AHP decision process, performed for the quantitative evaluation, involves pairwise comparisons commonly used in evaluation problems with a limited number of participants, since by augmenting the size of the participants beyond 15 there is no significant change in the final outcome [13]. As in our case there are 24 participants, it is very interesting to investigate the impact of modifying the group size of participants, from 24 to 15, on the AHP results. More specifically, it would be interesting to investigate the consistency of AHP results regarding the importance of elements towards the modelers intention to adopt CMMN, namely the results calculated with Eqs. 1 and 2. Such a verification would be very important as it is related with the objective of the evaluation, as it is projected in Fig. 5. In this context, we perform Monte Carlo simulations [51] of $N_{MC}=10^5$ iterations. For each iteration z ($z\le N_{MC}$) we randomly ignore a group of 9 participants and estimate the average weights of elements $W_k^{(z)}$ for the new group of $M=15$ participants. Finally, we estimate the average weights from all iterations $W_k$.

$$\begin{aligned} w_k= \frac{1}{N_{MC}}\sum _{z=1}^{N_{MC}}W_k^{(z)} \end{aligned}$$

(4)

Changes of elements and factors weights. In order to further validate the reliability of the final ranking of the factors for intention to adopt, given the level of uncertainties involved by carrying out a sensitivity analysis, Monte Carlo simulations are performed by simultaneously changing more than one parameters. The weights of all elements and the relative scores of factors are perturbed from $w_k$, $S_{ik}$ to $w_{k}(1+\varDelta w_k)$, $S_{ik}(1+\varDelta S_{ik})$, respectively, where the perturbations $\varDelta w_k$, $\varDelta S_{ik}$ are assumed zero mean, identically distributed, independent random variables uniformly distributed inside [$-s s$] [12]. Such random perturbation may be due to inconsistencies of the pairwise comparison matrices [53].