Deriving Tests with Guaranteed Fault Coverage for Finite State Machines with Timeouts

Abstract

In contrast to untimed FSMs, two minimal initialized FSMs with timeouts can be equivalent but not isomorphic. Accordingly, we propose an appropriate fault model and a method for complete test derivation for initialized deterministic FSMs with timeouts based on an appropriate FSM abstraction of the timed FSM specification. We also show how the same approach can be used for deriving tests for FSMs with both time guards and timeouts.

1 Introduction

A multitude of approaches are given for test derivation from formal specifications modeled as Finite State Machines (FSMs). The W method [1] paved the way for many derivatives to work on the test derivation considering various classes of FSM specifications and Implementations Under Test (IUT). For related summary and experiments the reader may refer to [2, 3]. Extensions to the W-based methods are also considered in the context of systems with timed constraints [4, 5]. Merayo et al. [6] establish a number of conformance relations for possibly non-deterministic FSM with input and output timeouts; however, test derivation is not considered in [6]. El-Fakih et al. [7] consider test derivation and assessment for timed FSMs with timed guards and single clock that is reset at every transition. Zhigulin et al. [8] presented a method for deriving complete test suites for FSMs with timeouts considering a traditional fault domain assuming that the number of states of an implementation TFSM does not exceed that of the reduced specification TFSM as well as the maximal finite timeout of the IUT does not exceed this of the specification. Recently, Bersolin et al. [9] investigated many timed FSM models with a single clock.

In this paper, we consider complete test derivation against FSMs with timeouts, hereafter denoted as TFSMs. In contrast to untimed FSMs, we show that two minimal initialized TFSMs can be equivalent but not isomorphic; moreover, we show that these TFSMs can have different number of states. According to [9], the behavior of a TFSM can be completely described by its corresponding (untimed) FSM abstraction and the reduced initially connected forms of corresponding FSM abstractions of two initialized equivalent TFSMs are isomorphic. This hints that the fault model and complete test derivation can be developed based on the reduced form of the FSM abstraction of a given TFSM specification. We consider complete test derivation with respect to an appropriate fault domain that contains every TFSM over the same input alphabet as the specification such that the reduced form of the FSM abstraction of an IUT has at most $m > 1$ states, and thus, the proposed approach is easily extended to FSMs with timeouts and timed guards.

2 Preliminaries

An initialized FSM is a 5-tuple $S = (S,I,O,h_{S} ,s_{0} )$ where I and O are input and output alphabets, S is a finite non-empty set of states with the designated initial state s₀, and $h_{S}\, \subseteq \,(S \times I \times O \times S)$ is the transition relation. We consider complete and deterministic FSMs, i.e., for each pair $\left( {s,i} \right) \in S \times I$ there exists exactly one transition $(s,i,o,s^{\prime}) \in h_{S}$. The equivalence and distinguishability relations between different states of FSMs are defined in a usual way [3]. It is known that given a complete deterministic initialized initially connected FSM, any two reduced initially connected forms of this FSM are isomorphic.

An FSM with timeouts, a TFSM for short, is an FSM annotated with a clock that is reset to zero at the execution of any transition. In addition, such a TFSM has input timeout transitions. When an input timeout expires at a state, the TFSM can spontaneously move to the destination state of the timeout transition while resetting the time to zero. An initialized TFSM is a 6-tuple $S = (I,S,O,h_{S} ,\Delta _{S} ,s_{1} )$ where I and O are input and output alphabets, S is the finite non-empty set of states, $h_{S} \,\subseteq\, S \times I \times O \times S$ is the transition relation and $\Delta _{S}$: $\Delta_{S} :S \to S \times (N \cup \{ \infty \} )$ is the timeout function, where N is the set of positive integers: for each state, this function specifies the maximum time for waiting for an input. Given state s of TFSM S such that $\Delta _{S} \left( s \right) \, = \, \left( {s^{\prime},T} \right)$, if no input is applied before the timeout T expires, S moves to state $s^{\prime}$ and the clock is set to zero. If $s = s^{\prime}$ then the clock is set to zero when timeout is expired. The transition $(s,i,o,s^{\prime}) \in S \times I \times O \times S$ means that S being at state s accepts an input i applied at time $t < T$ measured from the moment when the clock was reset at state s of S; the clock then is set to zero and S produces o. Hereafter, the timeout at state s can be written as $T_{s}$ or T when s is known from the context, for short.

TFSM S is a deterministic complete TFSM if for each pair $\left( {s,i} \right) \in S \times I$, there is exactly one transition $(s,i,o^{\prime},s^{\prime}) \in h_{S}$. In this paper, we consider only deterministic complete TFSMs. TFSM is (initially) connected if each state is reachable from the initial state. Given a TFSM S, a timed input is a pair (i, t) where $i \in I$ and t is a real; a timed input (i, t) means that input i is applied to the TFSM at time instance t where t is a local time. A sequence of timed inputs $\upalpha = \left( {i_{1} ,\,t_{1} } \right)\, \ldots \,\left( {i_{n} ,\,t_{n} } \right)$ is a timed input sequence. A sequence $\upalpha/\upgamma = \left( {i_{1} ,\,t_{1} } \right)/o_{1} \, \ldots \,\left( {i_{n} ,\,t_{n} } \right)/o_{n}$ of consecutive pairs of timed inputs and outputs starting at the state s is a timed trace of TFSM S at state s. Given complete deterministic TFSMs S and P, states s of S and p of P are equivalent if output responses at these states coincide for each timed input sequence; otherwise, s and p are distinguishable. Two initialized TFSMs S and P are equivalent if their initial states are equivalent. If any two different states of TFSM S are distinguishable then S is (state) reduced or minimal.

Consider two complete deterministic TFSMs in Fig. 1 which are equivalent. Each state in $S_{1} \,\left( {\text{a}} \right)$ and $S_{2} \,\left( {\text{b}} \right)$ is reachable from the initial state and both machines are reduced. However, these two equivalent machines are not isomorphic; moreover, they have different number of states.

Fig. 1.

Two equivalent yet not isomorphic TFSMs S₁ (a) and S₂ (b) and their FSM abstractions.

In order to calculate an output for a timed input (i, t) for each state s of TFSM S we consider the function $time\left( {s,t} \right) \, = s^{\prime}$ that determines state $s^{\prime}$ that will be reached by S through timeouts if no input was applied during t time units. The output response $\upbeta$ of S to a sequence $\upalpha = \left( {i_{1} ,\,t_{1} } \right)\left( {i_{2} ,\,t_{2} } \right)\, \ldots \,\left( {i_{n} ,\,t_{n} } \right)$ at state s is iteratively determined starting from state s.

Determining if two states of a TFSM S are equivalent or distinguishable can be done using the (untimed) FSM-abstraction A_S of S defined in [9].

FSM Abstraction:

Given a complete deterministic TFSM $S = (S,I,O,h_{S} ,\Delta _{S} ,s_{0} )$, we derive the FSM abstraction of S as the FSM $A_{S} = (S_{A} ,I_{A} ,O_{A} ,\uplambda_{AS} ,\left( {s_{0} , \, 0} \right)$, where $I_{A} = I \cup \{ 1\} ,O_{A} = O \cup \{ 1\}$. The input (output) 1 is a special input (output) of the FSM abstraction denoting the time duration. For each state s, the set S_A has a state (s, 0). Moreover, for each state s where the timeout T_s is finite, the set S_A has the states $\{ \left( {s,\,1} \right),..,\,(s,T_{s} {-} \, 1)\}$. Given state $\left( {s,\,t_{j} } \right) \in S_{A}$ of $A_{S}$ and input i, a transition $((s,t_{j} ),i,o, \, (s^{\prime},0))$ is a transition of the abstraction $A_{S}$ iff there exists a transition $(s,i,o,s^{\prime}) \in h_{S}$. Transitions under the input 1 correspond to timeout transition between states. Given state s such that $\Delta _{S} \left( s \right) = \left( {s^{\prime},T_{s} } \right)$ where $1 < T_{s} < \infty$, there are transitions $((s,0),1,1,(s,1)),\, \ldots ,\,((s,T_{s} - 2),1,1,(s,T_{s} - 1)), \, ((s,T_{s} - 1))$, in $\uplambda_{AS}$. If $\Delta _{S} \left( s \right) = \left( {s^{\prime},T_{s} } \right)$ then there is a transition $((s,T_{s} - 1),1,1,(s^{\prime},0))$ while there is a transition $((s,0),1,1,(s,0)) \in\uplambda_{AS}$ iff $T_{s} = \infty$. In [9], it is shown that the FSM abstraction of a complete and deterministic TFSM S is also complete and deterministic. As an example, consider the FSMs $S_{1}$ and $S_{2}$ in Fig. 1(a) and (b), their corresponding isomorphic FSM abstractions $A_{S1}$ and $A_{S2}$ are also shown in Fig. 1.

By definition, given an FSM with timeouts with n states and k inputs, the corresponding FSM abstraction has $\left( {k + 1} \right)$ inputs and the number of states of the FSM abstraction equals $\sum\nolimits_{{s \in S^{\prime}}} {(T_{s} + \left| {S\backslash S^{\prime}} \right|}$ where $S^{\prime}$ is the subset of all FSM states for which the timeout $T_{s}$ is finite.

A timed input sequence $\upalpha$ of TFSM S can be transformed into a corresponding input sequence $\upalpha_{FSM}$ of the FSM abstraction A_S. In this case, each timed input (i, t) is replaced by sequence $1.1 \, \ldots \,1$. i of inputs of the FSM abstraction where the number of inputs 1 equals t. At the same time the response of the FSM abstraction to sequence $1.1 \, \ldots \,1$. i is the sequence $1.1 \, \ldots \,1$. o where the number of outputs 1 is the same as for the timed input (i, t) and o is the response of the TFSM to timed input (i, t). Thus, the output sequence of the FSM abstraction $\upgamma_{FSM}$ is exactly the output sequence $\upgamma$ after removing all outputs 1. As there is no ambiguity, we further do not distinguish sequences $\upgamma_{FSM}$ and $\upgamma$.

Proposition 1.

Given a complete deterministic TFSM S and its corresponding FSM abstraction A_S, a timed trace $\upalpha/\upgamma$ exists for TFSM S if and only if there exists a trace $\upalpha_{FSM} /\upgamma$ for the FSM abstraction $A_{S}$.

Proposition 2 [9].

Two complete deterministic TFSMs are equivalent if and only if their FSM abstractions are equivalent.

The following proposition describes an input sequence that distinguishes two non-equivalent TFSMs.

Proposition 3.

Given two non-equivalent complete deterministic TFSMs S and P over the same input and output alphabets, let $A_{S}$ and $A_{P}$ be their FSM abstractions. If an input sequence $\upalpha_{FSM} = 1.1\, \ldots \,1.\,i_{1} \, \ldots \,1.1\, \ldots \,1$. i_k distinguishes FSM abstractions $A_{S}$ and $A_{P}$, then the timed input sequence $(i_{1} ,t_{1} )\, \ldots \,(i_{k} ,t_{k} )$ where $t_{j}$ is the number of inputs before the input $i_{j}$, $1 \le j \le k$, distinguishes machines S and P.

An FSM abstraction of a TFSM can be reduced using a traditional way. Then the FSM abstraction of a TFSM implementation can be compared with the FSM abstraction of the specification TFSM and if they are not equivalent then corresponding TFSMs can be distinguished by some input sequence $\upalpha_{FSM}$. Moreover, a corresponding timed input sequence α will distinguish the TFSM implementation from the specification TFSM (Proposition 3). Correspondingly, a complete test suite can be derived based on the minimal form of the FSM abstraction of the specification TFSM. Such a test suite is derived for timed sequences over local time and later we discuss how the test cases can be written over global time. We also note that when distinguishing two initialized deterministic complete FSMs $A_{S}$ and $A_{P}$, a distinguishing input can be only $i \in I$, as input 1 is defined at each state with the output 1. The sequence $\upalpha_{FSM}$. i distinguishes FSMs $A_{S}$ and $A_{\text{P}}$ and based on it a corresponding distinguishing sequence for TFSMs S and P can be constructed (Proposition 3).

When applying test cases to an IUT, we reasonably assume that each transition is performed with some small output delay θ such that the sum of all delays during a test case application is less than 1 and since timeouts are integers and these delays are very small they do not effect a proposed fault model.

3 Fault Models and Test Derivation

Given a specification TFSM S, we consider the fault model ${<}S, \cong ,FD_{m}{>}$, where $FD_{m}$ contains every TFSM P over the same input alphabet as S such that the reduced form of the FSM abstraction of P has at most $m > 1$ states. We note that it can well happen that some timed FSMs with less states than the specification TFSM are not included into the fault domain and vice versa a number of timed FSMs with more states than the specification TFSM are included into the fault domain.

Theorem 1.

The test suite TS obtained by Algorithm 1 is complete with respect to the fault model $< S, \cong ,FD_{m} >$.

4 Deriving Tests for FSMs with Timed Guards and Timeouts

In [9], FSMs with timed guards and timeouts are considered. Input timed guards describe the behavior at a given state for inputs, which arrive at different time instances. Formally, an initialized TFSM is a 6-tuple $S = (I,S,O,h_{S} ,\Delta _{S} ,s_{0} )$ where I and O are input and output alphabets, S is the finite non-empty set of states, $h_{S} \, \subseteq \,S \times I \times O \times S \times\Pi$ is the transition relation and $\Delta _{S}$ is the timeout function. The set $\Pi$ is a set of input timed guards. An input timed guard $g \in\Pi$ describes the time domain when a transition can be executed and is given in the form of interval ⌈min, max⌉ from [0; T), where ⌈ ∈ {(, [},⌉ ∈}, ), ]} and T is the value of the (input) timeout at the current state. The transition $(s,i,o,s^{\prime},g) \in S \times I \times O \times S \times\Pi$ means that TFSM S being at state s accepts an input i applied at time $t \in g$ measured from the moment when S entered state s; the clock then is set to zero and S produces output o. TFSM S is a deterministic complete TFSM if for each two transitions $\left( {s,i,o_{1} ,s_{1} ,g_{1} } \right),\left( {s,i,o_{2} ,s_{2} ,g_{2} } \right) \in h_{s}$ it holds that $g_{1} \cap g_{2} = \emptyset$ and the union of all input timed guards at state s under input i equals [0; T) when $\Delta _{S} \left( s \right) \, = \, \left( {s^{\prime},T} \right)$. Given a complete deterministic TFSM S, the largest finite boundary B_S of input timed guards and timeouts, we derive the FSM abstraction of S as the FSM $A_{S} (B) = (S_{A} ,I \cup \{ 1\} ,O \cup \{ 1\} ,\uplambda_{AS} ,(s_{0} , \, 0)),B \ge B_{S}$, where $S_{A} = \{ \left( {s,0} \right),\left( {s, \, \left( {0, \, 1} \right)} \right), \ldots ,\left( {s,\left( {B{-}1,B} \right)} \right),\left( {s,B} \right),(s, \, (B,\infty )):s \in S\}$. In [9], it is shown that such an FSM abstraction of a complete and deterministic TFSM S is also complete and deterministic and a timed input sequence α of TFSM S can be transformed into a corresponding input sequence $\upalpha_{FSM}$ of the FSM abstraction $A_{S} \left( B \right)$ similar to an FSM with timeouts. We then consider the fault model ${<}S, \cong ,FD_{m} (B){>}$, where $FD_{m} (B)$ contains every TFSM P over the same input alphabet as S such that the reduced form of the FSM abstraction of P has at most $m > 1$ states and the largest finite boundary of input timed guards and timeouts is $B \ge B_{S}$. In our case, the test derivation technique completely coincides with Algorithm 1 where the FSM abstraction $A_{S}$ is considered and the test suite TS obtained by Algorithm 1 is complete w.r.t. the fault model ${<}S, \cong ,FD_{m} (B){>}$.

5 Conclusion

A proper fault domain is considered for complete test derivation against timed FSMs. The fault domain takes into account the fact that a reduced TFSM specification and a reduced TFSM implementation with timeouts can be equivalent yet not isomorphic. A proper characterization of the fault domain is then considered using the unique reduced form of the FSM abstraction of the given timed FSM specification. The fault domain is extended to consider FSMs with timeouts and timed guards.