Debugging of Concurrent Systems Using Counterexample Analysis

Abstract

Model checking is an established technique for automatically verifying that a model satisfies a given temporal property. When the model violates the property, the model checker returns a counterexample, which is a sequence of actions leading to a state where the property is not satisfied. Understanding this counterexample for debugging the specification is a complicated task for several reasons: (i) the counterexample can contain hundreds of actions, (ii) the debugging task is mostly achieved manually, and (iii) the counterexample does not give any clue on the state of the system (e.g., parallelism or data expressions) when the error occurs. This paper presents a new approach that improves the usability of model checking by simplifying the comprehension of counterexamples. Our solution aims at keeping only actions in counterexamples that are relevant for debugging purposes. To do so, we first extract in the model all the counterexamples. Second, we define an analysis algorithm that identifies actions that make the behaviour skip from incorrect to correct behaviours, making these actions relevant from a debugging perspective. Our approach is fully automated by a tool that we implemented and applied on real-world case studies from various application areas for evaluation purposes.

1 Introduction

Concurrent and distributed applications are used in various domains, such as cyber-physical systems, software and middleware technologies, Service Oriented Computing, cloud computing, or the Internet of Things. The design and development of these applications is complex and cannot be achieved without introducing subtle bugs, which are defects of the software that prevent the correct behaviour of the system. The process of finding and resolving bugs is commonly called debugging. This process is a challenging task for a developer, since it is difficult for a human being to understand the behaviour of all the possible executions of this kind of systems, and bugs can be hidden inside parallel behaviours. There is a need for automatic techniques that can help the developer in detecting and understanding those bugs.

Model checking [8] is an established technique for verifying concurrent systems. It takes as input a model and a property. A model describes all the possible behaviours of a concurrent program and is produced from a specification of the system. In this paper, we adopt Labelled Transition Systems (LTS) as model description language. A property represents the requirements of the system and is usually expressed with a temporal logic. Given a model and a property, a model checker verifies whether the model satisfies the property. When the model violates the property, the model checker returns a counterexample, which is a sequence of actions leading to a state where the property is not satisfied.

Although model checking techniques automatically find bugs in concurrent systems, it is still difficult to interpret the returned counterexamples for several reasons: (i) the counterexample can contain hundreds (even thousands) of actions, (ii) the debugging task is mostly achieved manually (satisfactory automatic debugging techniques do not yet exist), and (iii) the counterexample does not give any clue on the state of the system (e.g., parallelism or data expressions) when the error occurs.

This work aims at developing a new approach for simplifying the comprehension of counterexamples and thus favouring usability of model checking techniques. In order to do this, we propose a method to produce all the counterexamples from a given model and to compare them with the correct behaviours of the model to better identify actions that caused the bug. The goal of our approach is to return as result an abstraction of counterexamples, which contains only those actions.

More precisely, we define a method that first extracts all the counterexamples from the original model containing all the executions. This procedure is able to collect all the counterexamples in a new LTS, maintaining a correspondence with the original model. Second, we define an analysis algorithm that identifies actions at the frontier between the new LTS and the original one. The frontier represents the area where counterexamples and correct behaviours, that share a common prefix, split in different paths. Actions at the frontier are relevant since they are responsible for the choice between a correct behaviour and a counterexample. We have implemented our approach in a tool and validated it on a set of real-world case studies from various application areas. Our experiments show that our approach is able to reduce the size of counterexamples by keeping only relevant actions at the frontier, and thus making the debugging process easier.

The rest of this paper is organized as follows. Section 2 introduces LTS models and model checking notions. Section 3 presents our counterexample abstraction techniques, including the generation of the LTS containing all the counterexamples and the process for identifying relevant actions in counterexamples. In Sect. 4, we describe our implementation and we apply it on real-word examples. Section 5 presents related work while Sect. 6 concludes this paper.

2 Preliminaries

In this work, we adopt Labelled Transition Systems (LTS) as behavioural models of concurrent programs. An LTS consists of states and labelled transitions connecting these states.

Definition 1

(LTS). An LTS is a tuple $M = (S, s^0, \varSigma , T)$ where S is a finite set of states; $s^0 \in S$ is the initial state; $\varSigma$ is a finite set of labels; $T \subseteq S \times \varSigma \times S$ is a finite set of transitions.

A transition is represented as $s \xrightarrow {l} s' \in T$, where $l\in \varSigma$. An LTS is produced from a higher-level specification of the system described with a process algebra for instance. Specifications can be compiled into an LTS using specific compilers. In this work, we use LNT as specification language [7] and compilers from the CADP toolbox [11] for obtaining LTSs from LNT specifications (see Sect. 4 for more details). However, our approach is generic in the sense that it applies on LTSs produced from any specification language and any compiler/verification tool. An LTS can be viewed as all possible executions of a system. One specific execution is called a trace.

Definition 2

(Trace). Given an LTS $M = (S, s^0, \varSigma , T)$, a trace of size $n \in ~\mathbb {N}$ is a sequence of labels $l_1, l_2, \ldots , l_n \in \varSigma$ such that $s^0 \xrightarrow {l_1} s_1 \in T, s_1 \xrightarrow {l_2} s_2 \in ~T, \ldots , s_{n-1} \xrightarrow {l_n} s_n \in T$. The set of all traces of M is written as t(M).

Note that t(M) is prefix closed. One may not be interested in all traces of an LTS, but only in a subset of them. To this aim, we introduce a particular label $\delta$, called final label, which marks the end of a trace, similarly to the notion of accepting state in language automata. This leads to the concept of final trace.

Definition 3

(Final Trace). Given an LTS $M = (S, s^0, \varSigma , T)$, and a label $\delta$, called final label, a final trace is a trace $l_1, l_2, \ldots , l_n\in \varSigma$ such that $s^0 \xrightarrow {l_1} s_1 \in T, s_1 \xrightarrow {l_2} s_2 \in ~T, \ldots , s_{n-1} \xrightarrow {l_n} s_n \in T$, $l_1, l_2, \ldots , l_n \ne \delta$ and there exists a final transition $s_{n} \xrightarrow {\delta } s_{n+1}$. The set of final traces of M is written as $t_{\delta }(M)$.

Note that the final transition characterized by $\delta$ does not occur in the final traces and that $t_{\delta }(M)\subseteq t(M)$. Moreover, if M has no final label then $t_{\delta }(M) = \emptyset$.

Model checking consists in verifying that an LTS model satisfies a given temporal property $\varphi$, which specifies some expected requirement of the system. Temporal properties are usually divided into two main families: safety and liveness properties [2]. In this work, we focus on safety properties, which are widely used in the verification of real-world systems. Safety properties state that “something bad never happens”. A safety property is usually formalised using a temporal logic (we use MCL [16] in Sect. 4). It can be semantically characterized by an infinite set of traces $t_{\varphi }$, corresponding to the traces that violate the property $\varphi$ in an LTS. If the LTS model does not satisfy the property, the model checker returns a counterexample, which is one of the traces characterised by $t_{\varphi }$.

Definition 4

(Counterexample). Given an LTS $M = (S, s^0, \varSigma , T)$ and a property $\varphi$, a counterexample is any trace which belongs to $t(M) \cap t_{\varphi }$.

Our solution for counterexample analysis presented in the next section relies on a state matching algorithm, which takes its foundation into the notion of preorder simulation between two LTSs [19].

Definition 5

(Simulation Relation). Given two LTSs $M_1 = (S_{1}, s^0_{1}, \varSigma _1, T_{1} )$ and $M_2 = (S_{2}, s^0_{2}, \varSigma _2, T_{2} )$, the simulation relation $\sqsubseteq$ between $M_1$ and $M_2$ is the largest relation in $S_1 \times S_2$ such that $s_1 \sqsubseteq s_2$ iff $\forall s_1 \xrightarrow {l} s'_1 \in T_1$ there exists $s_2 \xrightarrow {l} s'_2 \in T_2$ such that $s'_1 \sqsubseteq s'_2$. $M_1$ is simulated by $M_2$ iff $s^0_1 \sqsubseteq s^0_2$.

3 Counterexample Analysis

In this section, we describe our approach to simplify counterexamples. We first introduce the procedure to build an LTS containing all counterexamples (counterexample LTS), given a model of the system (full LTS) and a temporal property. We then present a technique to match all states of the counterexample LTS with states of the full LTS. This step allows us to identify transitions at the frontier between the counterexample and the full LTS. The frontier is the area where traces, that share a common prefix in the two LTSs, split in different paths. We define a notion of neighbourhood to extract sets of relevant transitions at the frontier and a procedure to collect the set of all neighbourhoods. Finally, by keeping transitions in these neighbourhoods, we are able to provide an abstraction of a given counterexample. To sum up, our approach consists of the four following steps, that we detail in the rest of this section:

• 1. Counterexample LTS generation

• 2. States matching

• 3. States comparison

• 4. Counterexample abstraction

3.1 Counterexample LTS Generation

The full LTS ($M_F$) is given as input in our approach and is a model representing all possible executions of a system. Given such an LTS and a safety property, our goal in this subsection is to generate the LTS containing all counterexamples ($M_C$).

Definition 6

(Counterexample LTS). Given a full LTS $M_F = (S_{F}, s^0_{F}, \varSigma _{F}, T_{F} )$, where $\delta \notin \varSigma _{F}$, and a safety property $\varphi$, a counterexample LTS $M_C$ is an LTS such that $t_{\delta }(M_C) = t(M_F) \cap t_{\varphi }$, i.e., a counterexample LTS is a finite representation of the set of all traces of the full LTS that violate the property $\varphi$.

We use the set of final traces $t_{\delta }(M_C)$ instead of $t(M_C)$ since $t(M_C)$ is prefix closed, but prefixes of counterexamples that belongs to $t(M_C)$ are not counterexamples. Moreover, traces in the counterexample LTS share prefixes with correct traces in the full LTS. Given a full LTS $M_F$ and a safety property $\varphi$, the procedure for the generation of the counterexample LTS consists of the following steps:

• 1. Conversion of the $\varphi$ formula describing the property into an LTS called $M_{\varphi }$, using the technique that allows the encoding of a formula into a graph described in [12]. $M_{\varphi }$ is a finite representation of $t_{\varphi }$, using final transitions, such that $t_{\delta }(M_{\varphi })=t_{\varphi } \cap \varSigma _{F}^{*}$, where $\varSigma _{F}$ is the set of labels occurring in $M_F$. In this step, we also apply the subset construction algorithm defined in [1] in order to determinise $M_{\varphi }$. We finally reduce the size of $M_{\varphi }$ without changing its behaviour, performing a minimisation based on strong bisimulation [17]. Those two transformations keep the set of final traces of $M_{\varphi }$ unchanged. The LTS $M_{\varphi }$ obtained in this way is the minimal one that is deterministic and accepts all the execution sequences that violates $\varphi$.

• 2. Synchronous product between $M_F$ and $M_{\varphi }$ with synchronisation on all the labels of $\varSigma _{F}$ (thus excluding the final label $\delta$). The result of this product is an LTS whose final traces belong to $t(M_F) \cap t_{\delta }(M_{\varphi })$, thus it contains all the traces of the LTS $M_F$ that violate the formula $\varphi$. Note that $t(M_F) \cap t_{\delta }(M_{\varphi }) = t(M_F) \cap t_{\varphi }$, because $t(M_F) \subseteq \varSigma _{F}^*$ and $t_{\delta }(M_{\varphi }) = t_{\varphi } \cap \varSigma _{F}^*$.

• 3. Pruning of the useless transitions generated during the previous step. In particular, we use the pruning algorithm proposed in [15] to remove the traces produced by the synchronous product that are not the prefix of any final trace.

Proposition:

The LTS $M_C$ obtained by this procedure is a counterexample LTS for $M_F$ and $\varphi$.

Let us illustrate this algorithm on the example given in Fig. 1. The full LTS on the left hand side represents a model of a simple protocol that performs send and receive actions in a loop. The counterexample LTS on the right hand side is generated with a property $\varphi$ stating that no more than one send action is allowed. Note that final transitions characterised by the $\delta$ label are not made explicit in the examples.

Fig. 1.

Full LTS and counterexample LTS

3.2 States Matching

We now need to match each state belonging to the counterexample LTS with the states from the full LTS. To do this, we define a matching relation between each state of the two LTSs, by relying on the simulation relation introduced in Sect. 2. In our context, we want to build such a relation between $M_C$ and $M_F$, where a state $x \in S_{C}$ matches a state $y \in S_{F}$ when the first is simulated by the latter, that is, when $x \sqsubseteq y$. Since the LTS that contains the incorrect behaviours is extracted from the full LTS, the full LTS always simulates the counterexample LTS. The algorithm that we have implemented to build the simulation between $M_C$ and $M_F$ relies on well-known graph traversal algorithms. More precisely, it relies on Breadth-First Search (BFS) to explore the graph. The algorithm is capable of performing backtracking steps in case it cannot match some states (this may happen due to nondeterministic behaviours present in both LTSs).

Let us consider again the example described in Fig. 1. Each state of the counterexample LTS on the right hand side of the picture matches a state of the full LTS on the left hand side as shown in Fig. 2. Note that multiple states of the counterexample LTS may correspond to a single state of the full LTS. In the example of Fig. 2, the property $\varphi$ has become unsatisfied after several iterations of the loop composed of Send and Recv actions, so that loop has been partially rolled out in the counterexample LTS, resulting in a correspondence of several states of the counterexample LTS to a single state of the full LTS.

Fig. 2.

States matching

It may also occur that a single state of the counterexample LTS may correspond to multiple states of the full LTS. For instance, the example given in Fig. 3 shows a full LTS and a counterexample LTS produced with a property that avoids Recv actions after a Send action. Thus, there exists a correspondence of more than one state of the full LTS with a single state of the counterexample LTS. In this specific case, the counterexample LTS can be described using a single trace, since the two states with an exiting Send transition after the Init transition simulate only one state in the counterexample LTS.

Fig. 3.

Multiple matching

3.3 States Comparison

The result of the matching algorithm is then analysed in order to compare transitions outgoing from similar states in both LTSs. This comparison aims at identifying transitions that originate from matched states, and that appear in the full LTS but not in the counterexample LTS. We call this kind of transition a correct transition.

Definition 7

(Correct Transition). Given an LTS $M_F = (S_{F}, s^0_{F}, \varSigma _{F}, T_{F} )$, a property $\varphi$, the counterexample LTS $M_C = (S_{C}, s^0_{C}, \varSigma _{C}, T_{C} )$ obtained from $M_F$ and $\varphi$, and given two states $s \in S_{F}$ and $s' \in S_{C}$, such that $s' \sqsubseteq s$, we call a transition $s \xrightarrow {l} s'' \in T_F$ a correct transition if there is no transition $s' \xrightarrow {l} s''' \in T_C$ such that $s''' \sqsubseteq s''$.

A correct transition is preceded by incoming transitions that are common to the correct and incorrect behaviours. We call these transitions relevant predecessors. Correct transitions allow us to introduce the notion of frontier. The frontier is a set of states at the border between the counterexample LTS and the rest of the full LTS, where for two matched states, there exists a correct transition in the full LTS.

Definition 8

(Frontier). Given an LTS $M_F = (S_{F}, s^0_{F}, \varSigma _{F}, T_{F} )$, a property $\varphi$, the counterexample LTS $M_C = (S_{C}, s^0_{C}, \varSigma _{C}, T_{C} )$ obtained from $M_F$ and $\varphi$, the frontier is the set of states $S_{ fr } \subseteq S_{F}$ such that for each $s \in S_{ fr }$, there exists $s' \in S_{C}$, such that $s' \sqsubseteq s$ and there exists a correct transition $s \xrightarrow {l} s'' \in T_F$.

A given state in the frontier allows us in a second step to identify a neighbourhood in the corresponding counterexample LTS, which consists of all incoming and outgoing transitions of that state.

Definition 9

(Neighbourhood). Given an LTS $M_F = (S_{F}, s^0_{F}, \varSigma _{F}, T_{F} )$, a property $\varphi$, the counterexample LTS $M_C = (S_{C}, s^0_{C}, \varSigma _{C}, T_{C} )$, two states $s \in S_{ fr }$ and $s' \in S_{C}$ such that $s'\sqsubseteq s$, the neighbourhood of state $s'$ is the set of transitions $T_{ nb } \subseteq T_{C}$ such that for each $t \in T_{ nb }$, either $t=s'' \xrightarrow {l} s' \in T_C$ or $t=s' \xrightarrow {l} s'' \in T_C$.

Let us illustrate these notions on an example. Figure 4 shows a piece of a full LTS and the corresponding counterexample LTS. The full LTS on the left hand side of the figure represents a state that is at the frontier, thus it has been matched by a state of the counterexample LTS on the right hand side and it has correct transitions outgoing from it. The incoming and outgoing transitions for this state in the counterexample LTS correspond to the neighbourhood.

Fig. 4.

Example of neighbourhood

3.4 Counterexample Abstraction

The final goal is to abstract a counterexample of the model in order to highlight the source of the bug and thus favour the comprehension of its cause. Given the counterexample LTS $M_C$, produced from a model $M_F$ and a property $\varphi$, where neighbourhoods have been identified in the previous subsection, and a counterexample $c_e$, produced from $M_F$ and $\varphi$, the procedure for the counterexample abstraction consists of the following steps:

• 1. Matching between states of $c_e$ with states of $M_C$.

• 2. Identification of states in $c_e$ that are matched to states in $M_C$, which belong to a neighbourhood.

• 3. Suppression of actions in $c_e$, which do not represent incoming or outgoing transitions of a neighbourhood.

Fig. 5.

Counterexample abstraction

For illustration purposes, let us consider the counterexample, produced by a model checker from a model M and a property $\varphi$, given on the top side of Fig. 5. Once the set of neighbourhoods in the counterexample LTS is computed using M and $\varphi$, we are able to locate sub-sequences of actions corresponding to transitions in the neighbourhoods. We finally remove all the remaining actions to obtain the simplified counterexample shown on the bottom side of the figure. We will comment on the relevance and benefit of these results on real-world examples in the next section.

4 Tool Support

In this section, we successively present the implementation of our approach, illustrate it on a case study, and present experimental results on examples found in the literature.

4.1 Implementation

Our tool is depicted in Fig. 6 and consists of two main parts. The first one implements the counterexample LTS generation step described in Sect. 3.1. It relies on the CADP toolbox [11], which enables one to specify and analyse concurrent systems using model and equivalence checking techniques. We particularly make use of the LNT value passing process algebra [7] for specifying systems, of the BCG binary format for representing LTSs, and of the MCL mu-calculus logic [16] for describing safety temporal properties. The LNT specification is automatically transformed into an LTS model in BCG format (the full LTS in Sect. 3) using CADP compilers. The CADP model checker (Evaluator [16]) takes as input an MCL property and an input specification/model (LNT or LTS), and returns a verdict (true or false + a counterexample if the property is violated). The computation of the counterexample LTS is achieved by a script we wrote using SVL [10], a scripting language that allows one to interface with tools provided in the CADP toolbox. This script calls several tools: a specific option of Evaluator for building an LTS from a formula following the algorithm in [12]; EXP.OPEN for building LTS products; Reductor for minimizing LTSs; Scrutator [15] for removing spurious traces in LTSs.

Fig. 6.

Overview of the tool support

The second part of our tool implements the algorithms for state matching (2), state comparison (3) and counterexample abstraction (4), described from Sects. 3.2 to 3.4. This part of the tool has been implemented in Java and consists of about 2,500 lines of code. The tool takes as input the files containing the full and the counterexample LTS, converted into an intermediate ASCII format called AUT (provided by CADP), and stores them in memory using a Java graph modelling library. The matching step (2) is based on a BFS graph search algorithm in order to build the simulation relation between the two LTSs. The state matching is then stored into a map, used by the state comparison step (3) to analyse outgoing transitions for each association of states between the two LTSs. This allows us to retrieve the set of neighbourhoods. Finally, the counterexample abstraction step (4) first produces the shortest counterexample from the full LTS and the property by using the Evaluator model checker, and second performs the counterexample reduction by locating and keeping actions that correspond to neighbourhoods. The result retrieved by our tool consists of the shortest counterexample abstracted in the form of a list of sub-sequences of actions, accompanied by the list of all neighbourhoods.

4.2 Case Study

We now describe an example taken from a real-world case study [20]. The example models a sanitary agency that aims at supporting elderly citizens in receiving sanitary agency assistance from the public administration. The model involves four different participants: (i) a citizen who requests services such as transportation or meal; the request can be accepted or refused by the agency; (ii) a sanitary agency that manages citizens’ requests and provides public fee payment; (iii) a bank that manages fees and performs payments; (iv) a cooperative that receives requests from the sanitary agency, receives payments from the bank, and provides transportations and meal services. Figure 7 gives the LTS model for each participant. We assume in this example that the participants interact together asynchronously by exchanging messages via FIFO buffers.

Fig. 7.

LTS models for the sanitary agency

Fig. 8.

Sanitary agency: full LTS and shortest counterexample

For illustration purposes, we use an MCL safety property, which indicates that the payment of a transportation service to the transportation cooperative cannot occur after submission of a request by a citizen to the sanitary agency:

We applied our tool to the sanitary agency model with the aforementioned property. Our tool was able to identify seven neighbourhoods in the couterexample LTS. The shortest counterexample involves three neighbourhoods, and this allows us to reduce its size from 19 actions to only 6 actions. Figure 8 shows (from left to right) the full LTS of the sanitary agency model, the shortest counterexample, and the three neighbourhoods (+ correct transitions) for this counterexample. The neighbourhoods and corresponding extracted actions are relevant in the sense that they precisely identify choices that lead to the incorrect behaviour. In particular, they identify the two causes of the property violation and those causes can be observed on the shortest counterexample. The first cause of violation is emphasized by the first neighbourhood and occurs when the citizen request is accepted. In that case, the refusal of the request is a correct transition and leads to a part of the LTS where the property is not violated. Indeed, when a citizen request is refused by the sanitary agency, the execution skips the part of the system behaviour where the transportation service and payment appear. The two next neighbourhoods pinpoint the second reason of property violation. They show that actions RECMONEYPOST_EM and PROVT_EM have been performed, which correspond to triggering the request for payment of the transportation service, that is not permitted by the property.

Our solution thus allows the developer to identify the cause of the property violation by identifying specific actions in counterexamples via the notion of neighbourhood. It is worth stressing that, since our approach applies on the counterexample LTS and computes all the neighbourhoods, the returned solution is able to pinpoint all the causes of the property violation, as we have shown with the example above.

4.3 Experimental Results

We carried out experiments on about 20 real-world examples found in the literature. For each example, we use as input an LNT specification or an LTS model as well as a safety property. Table 1 summarizes the results for some of these experiments. The first two columns contain the name of the model, the reference to the corresponding article, and the property. The third and fourth columns show the size of the full and the counterexample LTSs, respectively, in terms of number of states, transitions and labels. The following columns give the number of identified neighbourhoods, the size of the shortest (retrieved with breadth first search techniques) and of the abstracted counterexample, respectively. Finally, the last two columns detail the execution time for the counterexample LTS production, and for the matching and comparison algorithms (in seconds).

Table 1. Experimental results

First of all, we can see a clear gain in length between the original counterexample and the abstracted one, which keeps only relevant actions using our approach. There is one case (online stock broker, last row) in which our solution was not able to reduce the counterexample. This may occur in specific cases when the counterexample (the shortest here) does not exhibit any actions corresponding to transitions in a neighbourhood. In that particular situation, our abstraction techniques cannot help the developer in the identification of the cause of the property violation.

As far as computation time is concerned, the table shows that, for these examples, the time for producing counterexample LTSs is slightly longer than the time for computing the matching/comparison algorithms, which is very low (less than a second). The script for counterexample LTS computation is longer because it calls several CADP tools in sequence, which takes time.

5 Related Work

In this section, we survey related papers providing techniques for supporting the debugging of specifications and programs. LocFaults [5] is a flow-driven and constraint-based approach for error localization. It takes as input a faulty program for which a counterexample and a postcondition are provided. This approach makes use of constraint based bounded model checking combined with a minimal correction set notion to locate errors in the faulty program. This work focuses on programs with numerical statements and relies on a constraint programming framework allowing the combination of Boolean and numerical constraints. In addition, the authors do not explicitly describe the capacity of their solution for analysing concurrent programs.

Concurrency is explicitly taken into account in [3, 4]. In [3], the authors choose the Halpern and Pearl model to define causality checking. In particular, they analyse traces of counterexamples generated by bounded model checking to localise errors in hardware systems. In [4], sequential pattern mining is applied to execution traces for revealing unforeseen interleavings that may be a source of error, through the adoption of the well-known mining algorithm CloSpan [24]. This work deals with various typical issues in the analysis of concurrent models, for instance the problem of increasing length of traces and the introduction of spurious patterns when abstraction methods are used. CloSpan is also adopted in [13], where the authors applied sequential pattern mining to traces of counterexamples generated from a model using the SPIN model checker. By doing so, they are able to reveal unforeseen interleavings that may be a source of error. The approach presented in [13] is able to analyse concurrent systems and to extract sequences of events for identifying bugs, thus representing one of the closest results to our work. Reasoning on traces as achieved in [3, 4, 13] induces several issues. The handling of looping behaviours is non-trivial and may result in the generation of infinite traces or of an infinite number of traces. Coverage is another problem, since a high number of traces does not guarantee to produce all the relevant behaviours for analysis purposes. As a result, we decided to work on the debugging of LTS models, which represent in a finite way all possible behaviours of the system.

Another solution for localization of faults in failing programs consists in using testing techniques. As an example, [18] presents a mutation-based fault localization approach and suggests the use of a sufficient mutant set to locate effectively the faulty statements. This mutation analysis approach applies on C programs under validation using testing techniques whereas we focus on formal specifications and models being analysed using model checking techniques. In [23], the authors propose a new approach for debugging value-passing process algebra through coverage analysis. The authors define several coverage notions before showing how to instrument the specification without affecting original behaviours. This approach helps one to find errors such as ill-formed decisions or dead code, but does not help to understand why a property is violated during analysis using model checking techniques.

6 Conclusion

In this paper, we have proposed a new method for debugging concurrent systems based on the analysis of counterexamples produced by model checking techniques. First, we have defined a procedure to obtain an LTS containing all the counterexamples given a full LTS and a safety property. Second, we have introduced the notion of neighbourhoods corresponding to the junction of correct and erroneous transitions in the LTS, as well as an algorithm for computing them by comparing the full LTS and the LTS consisting of all counterexamples. Finally, we have implemented our approach as a tool and evaluated it on real-world case studies, showing the advantage of the counterexample abstraction in practice when adopting the neighbourhood approach.

As far as future improvements are concerned, a first perspective of this work is to extend our approach to focus on probabilistic specifications and models, and refine our LTS analysis techniques for handling those models. Another perspective is to increase the scope of system requirements that we can take into account. Indeed, although safety properties already allow us to define most requirements for real-world systems, we would like to consider liveness properties as well. Finally, we plan to investigate the introduction of code colouring in the specification by highlighting code portions that correspond to the source of the problem according to our approach.