Keywords

1 Introduction

Business Process Management (BPM) refers to specification, implementation, and monitoring of executable business processes [3].

By adopting BPM, organizations strive to reduce cost and cycle time while improving the quality of their products and services [11]. Such improvements are achieved by modeling business processes in an understandable way as a basis for analysis and redesign. A key concern is that the resulting process models have to be correct.

Modeling problems may lead to errors during process execution, which entails quality issues and additional costs for the organizations [4].

Various notations have been proposed for process modeling [7] (e.g., Event-Driven Process Chain, UML Activity Diagram, Business Process Model and Notation (BPMN), etc.) and a variety of process modeling tools [4] (e.g., Bonita, Camunda) is available. Recent years have seen a convergence on BPMN [9], and its version 2.0 has been adopted as an ISO standard.

Quality assurance has been less pervasive. Some tools provide functionality for detecting subclasses of problems and many problems go undetected depending on the tool [4]. Rozman et al. [10] identified problems being repeated constantly across a couple of thousands of processes modeled by novice modelers and defined a set of anti-patterns of business process modeling. Even professionally created process models such as the SAP Reference Model [6] contain errors.

Anti-patterns are partially useful not only for spotting such errors, but also by providing feedback to the modelers regarding which scenarios should be avoided in the future [10].

In this paper, we investigate the support of anti-patterns by BPMN-based process modeling and execution tools, in order to understand to which extent these tools automatically detect anti-patterns and help the user to correct them. We found that most common anti-patterns are detected by one or the other tool. Our analysis illustrates gaps in detection support. To achieve this analysis, we selected a set of anti-patterns and a set of BPMN 2.0-based process modeling and execution tools to be analyzed. To the best of our knowledge, there is no other paper examining process modeling tools from this point of view.

The organization of our paper is as follows. Section 2 presents the background, while Sect. 3 discusses related work. In Sect. 4 we discuss the methodology we adopted. Section 5 holds our result analysis report. Conclusions and future research are presented in Sect. 6.

2 Background

In this section, we present necessary concepts for understanding the contribution. First, we introduce BPMN 2.0 basic elements. Second, we present examples of anti-patterns selected for our tools analysis.

2.1 Business Process Model and Notation

One of the main purposes of process modeling is to ensure the repeatability of processes in organizations. The notation used to the modeling task of BPM is often the BPMN. There are several different elements in the BPMN to enable the process modeling task. For this paper understanding purposes, we present the process model example in Fig. 1, containing a key subset of core BPMN 2.0 elements needed for further discussion on this paper. This model sample represents the process of a store that sends an invoice to a client and waits for the client to realize the payment. If no payment is received up until 30 days, the main process flow is interrupted and the purchase is cancelled.

Fig. 1.
figure 1

Process model example illustrating a key subset of core BPMN 2.0 elements.

Full size image

2.2 Process Modeling Anti-patterns

For the process modeling task there are studies that aims at defining process modeling patterns to support process modeling reuse (e.g., [14]). Based on that, we wanted to identify a set of common problems encountered in the process modeling task and, thus, we conducted a literature research aiming to find works proposing anti-patterns in this area. As a result, we obtained the study of [10] which defines a set of fifteen anti-patterns based on the analysis of a couple of thousands of process models modeled by novice modelers along six years. The advantage of anti-patterns is that typically they come with a proposed solution to fix it, which was the case.

According to Rozman et al. [10], with the adoption of BPMN as the main notation for process modeling, the main commercial tools available for process modeling started adopting BPMN as standard notation. Although BPMN is the most complete notation available, it does not prevent the design of process models with problems which may, for example, prevent the achievement of cost reduction through process execution errors [4]. The process modeling problems, identified by Rozman et al. [10] within their process modeling anti-patterns, are basically of three types: (i) syntactic, regarding the misuse of BPMN notation elements; (ii) pragmatic, concerning the comprehensibility of the process models; and, (iii) semantic, characterized by the compliance of the process modeled and its real-world representative. In many cases, the process can be syntactically valid, however, its semantics may be dubious not only for the user who analyzes the process, but also for the engine that will execute it.

Following, we present two examples of anti-patterns modeled with BPMN 2.0 and based on Rozman et al [10]. For each example, we consider: (i) a description of the problem formulated as an anti-pattern; (ii) the impact caused by the anti-pattern being exposed; and, finally, (iii) a proposed solution represented by a correct process modeling pattern. These examples will be presented in Sect. 5.

Fig. 2.
figure 2

Anti-pattern activities in a pool not connected.

Full size image

Figure 2(a) represents the first anti-pattern example, concerning activities in a pool are not connected. This problem is characterized by no sequence flow connecting two activities inside the same pool, and its cause is a misinterpretation of modelers who consider multiple pools as a single process; or, consider that there exists a dependency between processes and that the use of the message flows connecting pools could dismiss the use of sequence flow in this case. The impact is that the relationship of dependence between the activities in Pool 2 (cf., Fig. 2(a)) is not defined, which could lead to the non-execution of “Task Y”, compromising the execution of the process, since this task will be unreachable. As a proposed solution (Fig. 2(b)), the process modeling should be done in such a way that the pools are independently modeled, that is, without taking into account the connections that exist between them. All process elements inserted inside a pool must be fully connected by sequence flows. Although pragmatically incorrect, reproducing this anti-pattern in the process modeling task is syntactically correct, according to BPMN 2.0; if a task does not have an output sequence flow (cf., “Task X” in Fig. 2(a)), this represents the end of one of the flow paths [8].

Figure 3 represents the second anti-pattern example, concerning exception flow is not connected to the exception. The problem is characterized by missing explicit exception flow. This is a semantic problem, since if any action was expected to be executed after the exception raises, nothing would be performed in this modeling. This type of modeling problem may cause the process to be interrupted with no compensation task executed after the exception. Besides, it may change the semantics of the process, especially when read by third-parties, hampering its comprehension, since it is not known if a sequence flow is missing, if it has been wrongly directly connected to the task to which it is attached, or if the process merely stops right after the exception is raised. As a proposed solution, the exception flow should be modeled to make explicit what the modeler desires to represent (e.g., Fig. 3(a)).

Fig. 3.
figure 3

Anti-pattern exception flow not connected to the exception.

Full size image

3 Related Work

We conducted a preliminary research of the literature, aiming at identifying approaches that deal with the same subject explored in our paper. Regarding anti-patterns, beyond the study presented by Rozman et al. [10], addressing the identification of common mistakes made in the process modeling task, there is the study conducted by [1]. In this paper, the authors focus on the analysis and verification of 14 anti-patterns in collaborative business processes between organizations. The models were implemented using the UML Profile for Collaborative Business Processes based on Interaction Protocols (UP-ColBPIP), a language for collaborative process modeling that extends UML2 semantics. A set of 10 anti-patterns for control flow of the UP-ColBPIP language were defined, and a tool for checking the process models to identify these anti-patterns was developed. Although, there were no practical application for comparison of the behavior of different process modeling tools concerning any of these anti-patterns, which differs these studies from ours.

Regarding the comparison of process modeling tools, we found approaches on the topic of good practices in process modeling, also known as guidelines, focusing on the use of BPMN. However, these approaches aim at defining good practices in process modeling and how a set of available tools on the market encourage these good practices; or, how these tools react if the user does not follow their good practices recommendation on the process modeling task. The approach presented by Snoeck et al. [12] is an example of this kind of study. The authors explore to what extent BPMN-based tools support a set of process modeling guidelines. Another study that compares process modeling tools is the one by [2]. In this work, the authors compare the performance of the tools focusing on the interoperability among these applications. These studies differ from ours, since we compare the behavior of tools regarding anti-patterns, not good practices for the process modeling task or interoperability among the tools.

In [13] the authors analyze the presence of “syntactic anomalies” and “structural anomalies” in business processes modeled with BPMN. Syntactic anomalies were set into three groups: incorrect use of flow objects, misuse of connection objects, and incorrect use of swimming pools. While the structural anomalies were set into four groups: deadlock, lack of synchronization, dead activity, and infinite loop. However, the study does not apply these process models anomalies to process modeling tools nor process execution simulation tools to verify their effects, which differs this study from ours too.

4 Research Methodology

The aims of our investigation on the behavior of a subset of leading tools regarding a subset of process modeling problems is to identify to which extent these tools, which handle processes modeled using the BPMN 2.0 standard, automatically detect anti-patterns and help the user to correct them. Besides, we want to identify if these tools behave similarly or differently to each other when dealing with the same process modeling problems. Since BPMN 2.0 is an ISO standard, we expect that the tools behave similarly. In this section, we describe the steps we follow to perform the tool analyses regarding their behavior in relation to anti-patterns for process modeling problems.

4.1 Selection of the Set of Process Modeling Anti-patterns

After conducting the literature research to search works dealing with common problems encountered in the process modeling task, we identified some studies [5, 10, 13, 15]. They point out to common problems encountered in process modeling tasks, some of them are the same across multiple studies (e.g., activities in a pool are not connected). But only the works in [5, 10, 15] explicitly define anti-patterns. Since [10] presents a more extensive set of anti-patterns, we mainly based our selection in their study, choosing the anti-patterns that also appear in the other papers that also list common problems encountered in process models [5, 13, 15]. Thus, we selected the ten anti-patterns: (1) Activities in one pool are not connected; (2) The event does not contain an end event; (3) Sequence flow crosses sub-process boundary; (4) Sequence flow crosses pool boundary; (5) Gateway receives, evaluates and sends a message; (6) Intermediate events are placed on the edge of the pool; (7) Hanging intermediate events or activities in the process model; (8) Each swimlane in the pool contains start event; (9) Exception flow is not connected to the exception; (10) Message flow misuse across swimlanes.

4.2 Selection of the Set of Process Modeling Tools

After selecting the anti-patterns subset, we selected a set of process modeling and execution tools to submit each of the selected anti-patterns and, thereafter, analyze each tool behavior regarding each process modeling problem. We defined the following inclusion criteria to conduct our tools selection process: (i) the application must have an open source version or a free trial version; (ii) the application must be recognized in the market as one of the references in BPM, considering results from literature and the tools’ main clients as parameters; (iii) the application should have recently updated version; (iv) the application, besides providing the necessary resources for the modeling of the processes, should enable execution simulation, so that it is possible to analyze its behavior when executing a process with process modeling anti-pattern; and, finally, (v) the application must support BPMN 2.0.

Moreover, we performed a literature research on modeling and execution tools. We found a variety of papers [4, 12] and based our selection on their results. The study [12], for example, which is a related work as presented in Sect. 3, made an exhaustive modeling tools analysis considering 117 tools. Thus, we selected the following process modeling and execution tools, also considering their acceptability in industry and academia communities: Bizagi (version 3.1), Bonita (version 7.5.4), Camunda, (version 1.10), Signavio, (version 11.9.1).

4.3 Tool Analysis Method Definition

In order to answer how the selected tools for process modeling and execution behave regarding process modeling anti-patterns, the following procedure was defined. First, each anti-pattern was modeled in BPMN 2.0 using each one of the selected tools, following an identical process of process modeling on each tool. All of the modeled and analyzed anti-patterns in our study are available at GitHubFootnote 1. Second, we analyzed how the tool behaves regarding the process model containing the anti-pattern. In particular, we wanted to know if: the tools emit any warning or error message to the user in the process modeling task, if the tools emit any warning message pre-execution simulation, and if the tools simulate the execution of the process model containing the anti-pattern. Third, we reported whenever a warning or error message was given to the user (independent of how the visual feedback was composed) regarding any problem detected by the tool in the anti-pattern; we also report whenever an explanation about the modeling problem or correction suggestion was given. Fourth, we verified whether the tool allowed the execution of the process model, how was the behavior of the tool when we tried to execute a process model containing the anti-pattern, and if any warning message was given to the user at runtime. Finally, the data collected from the analysis of each tool was collected and reported in a comprehensive manner, further presented in Sect. 5.

5 Tool Behavior Analysis Concerning Anti-patterns

In this section, we provide an overview of the overall results (Sect. 5.1), followed by a more detailed discussion of two illustrative examples (Sects. 5.2 and 5.3, respectively). Finally, we discuss the overall anti-pattern detection support in the four tools (Sect. 5.4).

5.1 Overall Results

After the analysis of the behavior of every tool regarding each anti-pattern chosen, we were able to build Table 1, which summarizes our work and enables us to point out our findings.

Table 1. Results of the tool behavior analysis concerning the modeling and execution tasks regarding each anti-pattern. For the modeling task we analyzed if the tool emitted (+) or not (−) any warning or error message to the user; while for the simulation task we analyzed if the tool emitted (+) or not (−) any warning, and if the tool executed (+), executed partially (±) or did not execute (−) the anti-pattern.
Full size table

On the process modeling phase, Bonita was the tool that most issued warnings (in 40% of the cases), followed by Signavio (30%), and Bizagi (10%). Camunda was the only tool that did not issue any warning regarding any of the anti-patterns analyzed. Considering all tools, warnings were issued to the user in 60% of the cases, more precisely, in processes modeled containing the anti-patterns 1, 2, 5, 6, 7, and 8. Regarding error messages in the process modeling phase, Signavio was the tool that most emitted error messages to user, preventing the modeling of processes containing anti-pattern in 60% of cases. Followed by Bizagi and Bonita (30%), and Camunda (10%). Considering all tools, error messages were issued for 90% of the anti-patterns analyzed (i.e., only one anti-pattern did not generate error messages: anti-pattern 3). The only anti-pattern we could not model was anti-pattern 5. Although, some tools emitted warning or error in the modeling phase.

Fig. 4.
figure 4

Comparison of the behavior of each tool regarding the anti-patterns.

Full size image

In general, tools that issued a warning regarding a modeling problem did not issue an error about the same problem, and vice-versa. Signavio was an exception, since it issued a warning and an error to anti-pattern 7. Considering this, Signavio was the tool that most reacted to process models containing anti-patterns, issuing warning or error in 80% of the cases analyzed, followed by Bonita (70%), Bizagi (40%), and Camunda (10%) (cf., Fig. 4).

On the simulation execution phase, only Camunda issued an error, precisely when it simulated a process model containing the anti-pattern 8. Moreover, Table 1 also shows that it was possible to simulate the execution of processes comprising anti-patterns in 90% of the cases (i.e., only one anti-pattern did not enable the execution simulation: anti-pattern 3). Considering all the anti-patterns that allowed the execution simulation to occur, in 44% of cases the tools performed this procedure showing distinct semantics, that is, the processes were interpreted differently across one or more tools. In 70% of the cases, more than one tool allowed the simulation of the execution of the processes.

Figure 4 shows a comparison of the behavior of each tool analyzed along our study. Bizagi was the tool that most allowed the simulation of the execution of processes containing anti-pattern, executing the simulation in 80% of the cases, followed by Bonita and Camunda (60%), and Signavio (30%).

5.2 Illustrative Example: Anti-pattern 1

As the first illustrative example of anti-pattern analyzed, we present the anti-pattern regarding activities in one pool are not connected (cf., Fig. 2(a)). The behavior of the tools are described below.

Bizagi did not issue any warning to the user regarding the anti-pattern in the process modeling task. When executing the process validation function, the tool stated that the process was validated “without warnings or errors”. Bizagi allowed the simulation of the execution of the process, and when the execution was initiated in Pool 1, the process passed through the pool activities and ended normally. However, upon initialization of the execution in Pool 2, the process terminated abruptly, right after the end of Task X, and did not perform Task Y.

Bonita issued an information-level warning to the user in the process modeling task, stating that there was no sequence flow on the input of Task Y and, for this reason, the activity would be used as a “possible” initialization activity. Upon the initialization of the process execution simulation through Pool 1, the process was executed and finalized normally. However, when starting through Pool 2, only Task X was executed. Task Y stayed pending, waiting to receive a message from Pool 1, since this task has no input sequence flow. When the message from Pool 1 is received, the process in Pool 2 ends normally.

Camunda has not issued any warning or error to the user about the anti-pattern present during the process modeling task. This tool allowed the simulation of the execution of the process. When beginning the execution simulation in Pool 1, the process passed through the activities of the pool and ended normally. However, upon initialization in Pool 2, the process terminated abruptly, after the end of Task X, and did not perform Task Y.

Signavio pointed to two errors when validating the process model in the process modeling task. One of them, referring to Task X, since it did not contain an output sequence flow; and, another error referring to Task Y, since it did not contain an input sequence flow. Due to these errors, the tool did not allow the execution of the process containing this anti-pattern.

5.3 Illustrative Example: Anti-pattern 9

As second illustrative example, we present the anti-pattern regarding Exception flow is not connected to the exception (cf., Fig. 3(a)). The exception flow, in this case, is the flow that should be executed right after the triggering of the interruptive timer intermediate event, attached to the activity, subsequently to the passage of a certain amount of time. The behavior of the tools regarding this anti-patterns are described next.

Bizagi did not allow the process to be modeled without a sequence flow connected to the intermediate event. When attaching an intermediate event to an activity, Bizagi automatically inserts a new sequence flow starting from the intermediate event. If nothing else is modeled from this intermediate event when performing the process validation, Bizagi issue an error message informing that “transition is not connected”. In this scenario, it is not possible to simulate the process execution.

Bonita did not allow the modeling of the process containing the anti-pattern. When executing the automatic validation of the model, function provided by the tool, which searches for syntactic errors within the model, Bonita issues an error message informing that “the edge event must have an exception transition”. Even so, the process execution simulation option is available. However, when trying to simulate the execution, the tool re-evaluates the model and prevents its simulation.

Camunda allowed the modeling of the process containing the anti-pattern and did not issue any warning or error message to the user. It also allowed the execution simulation without the emission of any warning.

Signavio did not allow the process containing the anti-pattern to be modeled. Upon performing the automatic validation of the model, provided by the tool, it issues an error message informing that “attached intermediate events must have exactly one output stream”. Although this tool has several levels of validation, the process with this anti-pattern was not even approved in the most basic validation. Thus, Signavio did not allow the execution simulation to be executed.

5.4 Discussion on the Overall Anti-pattern Detection Support in the Selected Tools

After conducting our tool analysis, our main finding is that distinct process modeling and execution tools behave differently regarding the same anti-pattern modeled with BPMN 2.0 (cf., Table 1), which is a standard notation and, therefore, should trigger similar responses by the tools concerning the same process models.

We point out to the need for the manufacturers of process modeling tools to invest in features to visually feedback and guide modelers not only to the adoption of best practices in the process modeling task but, also, in the warning of known anti-patterns found in processes. Although some tools are already investing in this issue (e.g., Camunda, Signavio), none of the tools analyzed in this study was efficient in this area to avoid the occurrence of the most well-known process modeling problems. Therefore, we believe our findings will inspire tool developers to give more attention to this issue.

Because BPMN 2.0 is consolidated as a standard notation for the process modeling task, it is usual for many users to model their processes in one tool and execute them in another. In this scenario, an analyst could validate a process modeled on one tool but, when executing the same process in another tool, get a different result than expected. As the results of our study point out, this kind of scenario may be very problematic for process execution. Another possible problematic scenario could be an organization changing its BPMN 2.0-based tool supplier. In this case, BPMN 2.0 being a standard induces the stakeholders to think that there is full compatibility across different process modeling and execution tools. However, our results shows that this is not always the case.

6 Conclusions and Future Research

In this paper, we presented an analysis of the behavior of process modeling tools during the modeling and execution (whenever possible) of processes modeled containing some of the most common anti-patterns practiced by users. After modeling and executing these processes on the four selected BPMN 2.0-based tools, we found out that the way manufacturers interpret the BPMN 2.0 notation and visually feedback users about process modeling problems differs from each other. Currently, these tools present simple and passive validation mechanisms based almost entirely on labeling good practice and basic checking (i.e., missing start or end events), being Signavio the tool that presented the highest levels of process validations, although this case study shows that there is still room for the tool to evolve further.

As future research we intend to increment the selected set of process modeling anti-patterns, and even design patterns, to analyze over the toolset; and, we intend to apply a survey with participants from different countries aiming at the identification of more BPMN 2.0-based tools used for modeling and executing of processes. As a limitation of our work, it is important to highlight that the processes modeled in our study were extremely simple, once they consisted of few tasks (i.e., 5 or less in most cases). In real-world situations, the complexity of the models is much higher, what makes this difference of behavior between the tools even more laborious to analyze.