Using PRISM model checker as a validation tool for an analytical model of IEEE 802.15.4 networks
Introduction
Performance models of networks and protocols are typically validated by using a well-known simulation framework, such as, for example, OPNET Modeler (now Riverbed Modeler), ns-2, or OMNeT++, or with a dedicated simulation tool developed from scratch. In [1], as well as in [2], an analytical model for performance analysis of the nonbeacon-enabled mode of the IEEE 802.15.4 Medium Access Control (MAC) protocol is proposed. A star-shaped wireless sensor network is considered which consists of a central node (a “sink”) surrounded by sensor nodes. It is assumed that all the sensor nodes receive a query from the sink simultaneously and that each of them tries immediately to send one message to the sink by executing the version of the mentioned MAC protocol without acknowledgements. The model consists of mathematical expressions for calculation of the probability of successful receipt of the message at a certain time and similar events. An algorithm is provided for calculation with these expressions. In order to validate the expressions, the authors have written a dedicated simulation tool in C and compared the simulation results for several combinations of parameters with the analytical results calculated by using the algorithm.
Over the last decade, formal methods and tools which enable quantitative verification have been developed intensively [3]. The most important quantitative verification technique developed is probabilistic and real-time model checking [4], [5]. Basically, model checking takes as input a state-machine model of a finite-state system, a specification of a property in the form of logical formula, and finds a definite answer as to whether the system has that property or not by automatically checking all its possible behaviors. In the past, model checking could only be used for the verification of logical correctness properties of systems, such as, for example, whether an event always happens after another one or whether a variable always has a particular value. With the invention of probabilistic model checking, it can also be used for verifying whether there is a certain probability of an event in the system, and, for example, to find out the probability of a certain event, the probability of an event at a given time, or the expected time until it happens. Having this capability, model checking has proved useful for the performance analysis of systems [6]. In the continuation, by formal verification we will mean model checking. Formal verification has several advantages over simulation. One of them is that all the possible runs of the system concerned are checked and the exact value of the probability or another quantity being sought is returned as a result [6]. Another advantage is that the property of interest can be checked automatically by expressing it by a logical formula and “pressing the button”, whereas simulators typically require the user to carry out some calculations or observations on the outcomes of the simulation by herself/himself, or to make them a part of the simulation code in order to obtain the result [7], [8]. Yet another advantage is that logical correctness properties can be stated with a logic and verified automatically before the verification of performance properties, whereas exhaustive verification of whether the model is built correctly is generally impossible by using the usual simulation frameworks.
Unfortunately, the probabilistic and real-time verification can usually only be performed for small systems because of the state-space explosion problem. It is for this reason that so-called statistical model checking is increasingly being investigated [9]. This technique is similar to simulation in that it executes the system model randomly up to a certain number of runs, but has an important advantage over simulation that properties of interest, including the correctness ones, can still be expressed by logical formulas, since it evaluates them on these runs. As, generally, not all the possible runs are examined, the answer obtained for a formula might be an approximation of the accurate result.
Probabilistic model checking has already been employed for the performance analyses of many kinds of networks and protocols (e.g., [10], [11], [12]), but to our knowledge, it has rarely been used for the validation of analytical models thereof. [13] reports that by using the probabilistic model checker PRISM [14], deficiencies have been found in an analytical web server performance model with proxy cache server. In [15], the results of using PRISM for probabilistic verification of a shuffling protocol for sharing data in a distributed network are compared to the results obtained with an analytical equational model.
PRISM is a powerful tool which supports formal modeling of networks with different kinds of state machines, enables the normal as well as statistical model checking, and can present quantitative results obtained by model checking on graphs [16]. In [17], [18], [19], it is employed for the probabilistic verification of a nonbeacon-enabled version of the IEEE 802.15.4 MAC protocol executed in similar kinds of networks as in [1], but without the aim of using the results to validate an analytical model of the protocol. The purpose of these works has been to devise accurate models of the standard protocol (see, e.g., [20]) and verify some performance properties. [17] and [18] present formal specifications of the networks with two sending nodes by using Markov Decision Processes (MDP) and Probabilistic Timed Automata (PTA) supported by PRISM, but do not provide accurate modeling of the clear-channel assessment period. Besides the improvement of the latter, [19] proposes different approaches to the formal specification of IEEE 802.15.4 star-shaped networks with an arbitrary number of sensor nodes by using PTAs. In [21], the effect of a non-standard clear-channel assessment period length in star-shaped 802.15.4 networks containing hidden nodes is analyzed with PRISM by using statistical model checking.
In contrast to [17], [18], [19], [21], the aim of the research presented in this paper was to try to use PRISM for validation of an analytical model. In particular, the aim was to take the analytical model from [1] as given and to see whether it was possible to use the model checking with PRISM as an alternative to the validation of this model by classical simulation. As is often the case (cf., e.g., [22], [23]), in [1] a MAC protocol slightly different from the standard one is considered, and different performance metrics are dealt with than in [17], [18], [19], [21]. That is why we could not just have used the models and formulas from those papers. The main contribution of this paper are two kinds of network models for PRISM which represent exactly the networks considered in [1], allow us to specify all the performance metrics from the analytical model with probabilistic formulas in PRISM in a simple way, and enable efficient normal, i.e., exhaustive, and, respectively, statistical model checking of them. Note that there is no reason to doubt the claims of [1] about the validity of the analytical model based on the results of simulation. The aim of this paper is, therefore, not to carry out the validation of that model again, but to show how PRISM could be used for its validation advantageously.
In Section 2, we provide a description of the protocol from [1]. In Section 3, we briefly present the mathematical model of the protocol as well as the performance metrics from [1]. In Section 4, we indicate how the considered protocol differs from the standard one and explain both kinds of network models written in PRISM for the former. In Section 5, it is shown how to specify each performance metric in order to be able to use the model checking for it. In Section 6, we present some results of the normal as well as statistical model checking for these metrics. Section 7 contains a discussion and concludes the paper.
Section snippets
MAC protocol
In [1], a nonbeacon-enabled star-shaped wireless personal area network is assumed consisting of a coordinator and N sensor nodes. The analytical model covers the behavior of the nodes after they receive a query from the coordinator. At that moment, “time 0”, each node starts to execute the following unslotted Carrier Sense Multiple Access with Collision Avoidance (CSMA-CA) algorithm in order to send one data frame directly to the coordinator.
The algorithm is based on units of time called
Analytical model
In this subsection, we present the analytical model from [1] to the extent needed to understand the rest of this paper. As in the MAC protocol considered in [1], the backoff delay, the duration of sensing, as well as the duration of transmission are expressed in units of time with duration db, the resolution time of the model is set equal to db, such a unit is called a slot, and the current time is expressed as the current slot number. The jth slot, for means the time from j · db to
Network specification in PRISM
In PRISM, a system specification consists of modules, which represent concurrent processes [16]. The syntax and semantics of the modules depend on the model type chosen. A difference between the MAC protocol considered in [1] and the standard one considered in [17], [18], [19], [21] is in that the clear channel assessment in the latter does not last a whole backoff period, i.e., 20 symbol times, but only 8 symbol times. If the node assesses the channel to be clear for the whole CCA period, it
Property specification in PRISM
In this section, we show that all the performance metrics listed in Section 3 can be expressed in PRISM’s property specification language. The latter subsumes several probabilistic temporal logics, including PCTL (Probabilistic Computation Tree Logic) [16]. For DTMC models, the queries about probabilities can be specified with formulas of the form P=? [pathprop], where pathprop is an LTL-style (LTL is short for Linear-time Temporal Logic) formula expressing a property of a path in the model. In
Some results of model checking
In this section, we present some results of model-checking the models with signaling with PRISM. In contrast to the simulation, it is possible to obtain exact numerical values for the performance metrics by using the normal model checking in PRISM to the extent allowed by the available computer. We, therefore, first provide some graphs showing the exact results. For comparison, we provide the results obtained with the analytical model on the same graphs.
We implemented the algorithm for the
Conclusions
We showed how PRISM could be applied successfully for the validation of the analytical performance model derived in [1] instead of writing a special simulation program for that purpose. We represented the protocol from [1] in a way that allowed us to query about all the probabilities by referring directly to the slot number, and to obtain the results quickly with normal model checking for small networks and with statistical model checking for larger ones. Instead of adapting an existing PTA or
Acknowledgements
This work was partially funded by the Slovenian Research Agency (ARRS) [contract number P2-0069].
References (25)
- et al.
Probabilistic model checking of the CSMA/CD protocol using PRISM and APMC
Electron. Notes Theor. Comput. Sci.
(2005) - et al.
Performance analysis of IEEE 802.15.4 non beacon-enabled mode
IEEE Trans. Veh. Technol.
(2009) - et al.
Performance analysis of the IEEE 802.15.4 MAC protocol
Sensor Networks with IEEE 802.15.4 Systems
(2011) - G. Norman, D. Parker, Quantitative verification: Formal guarantees for timeliness, reliability and performance, a...
Perspectives in probabilistic verification
Proc. 2nd IFIP/IEEE Int. Symp. Theoretical Aspects of Software Engineering (TASE)
(2008)- et al.
Verification of real-time probabilistic systems
- et al.
PRISM: probabilistic model checking for performance and reliability analysis
Perf. E. R.
(2009) - OMNeT++ Simulation Manual, https://omnetpp.org/doc/omnetpp/manual/ accessed...
- OPNET: Manual de usuario, Universitat Politècnica de Catalunya, Departament d’Enginyeria Telemàtica, Secció de...
- et al.
Statistical model checking: An overview
Proc. 1st Int. Conf. Runtime Verification (RV)
(2010)