Using PRISM model checker as a validation tool for an analytical model of IEEE 802.15.4 networks

https://doi.org/10.1016/j.simpat.2017.08.002Get rights and content

Abstract

Commonly, simulation by using an existing network simulation tool or a simulator developed from scratch is employed for validation of analytical network performance models. An analytical model of star-shaped wireless sensor networks has been proposed in the literature in which, upon receiving a query from the coordinator, each sensor node sends one data frame to it by executing the IEEE 802.15.4 unslotted carrier-sense multiple access with collision avoidance algorithm. The model consists of expressions for calculation of the probability of successful receipt of the data at a certain time and the like. The authors of the model have written a special simulation program in order to validate the expressions. Our aim was to employ probabilistic model checker PRISM instead. PRISM only requires the user to formally specify the network as a kind of state machine and the queries about the probabilities sought in the form of logical formulas. It finds the probabilities automatically and can present them on graphs. We show how to specify the networks formally in such a way that all the expressions from the analytical model can be validated with PRISM. For those networks containing a few nodes, the validation can be carried out by normal model checking, which, in contrast to the simulation, always checks all the possible network behaviors, whereas statistical model checking can be used for the larger networks.

Introduction

Performance models of networks and protocols are typically validated by using a well-known simulation framework, such as, for example, OPNET Modeler (now Riverbed Modeler), ns-2, or OMNeT++, or with a dedicated simulation tool developed from scratch. In [1], as well as in [2], an analytical model for performance analysis of the nonbeacon-enabled mode of the IEEE 802.15.4 Medium Access Control (MAC) protocol is proposed. A star-shaped wireless sensor network is considered which consists of a central node (a “sink”) surrounded by sensor nodes. It is assumed that all the sensor nodes receive a query from the sink simultaneously and that each of them tries immediately to send one message to the sink by executing the version of the mentioned MAC protocol without acknowledgements. The model consists of mathematical expressions for calculation of the probability of successful receipt of the message at a certain time and similar events. An algorithm is provided for calculation with these expressions. In order to validate the expressions, the authors have written a dedicated simulation tool in C and compared the simulation results for several combinations of parameters with the analytical results calculated by using the algorithm.

Over the last decade, formal methods and tools which enable quantitative verification have been developed intensively [3]. The most important quantitative verification technique developed is probabilistic and real-time model checking [4], [5]. Basically, model checking takes as input a state-machine model of a finite-state system, a specification of a property in the form of logical formula, and finds a definite answer as to whether the system has that property or not by automatically checking all its possible behaviors. In the past, model checking could only be used for the verification of logical correctness properties of systems, such as, for example, whether an event always happens after another one or whether a variable always has a particular value. With the invention of probabilistic model checking, it can also be used for verifying whether there is a certain probability of an event in the system, and, for example, to find out the probability of a certain event, the probability of an event at a given time, or the expected time until it happens. Having this capability, model checking has proved useful for the performance analysis of systems [6]. In the continuation, by formal verification we will mean model checking. Formal verification has several advantages over simulation. One of them is that all the possible runs of the system concerned are checked and the exact value of the probability or another quantity being sought is returned as a result [6]. Another advantage is that the property of interest can be checked automatically by expressing it by a logical formula and “pressing the button”, whereas simulators typically require the user to carry out some calculations or observations on the outcomes of the simulation by herself/himself, or to make them a part of the simulation code in order to obtain the result [7], [8]. Yet another advantage is that logical correctness properties can be stated with a logic and verified automatically before the verification of performance properties, whereas exhaustive verification of whether the model is built correctly is generally impossible by using the usual simulation frameworks.

Unfortunately, the probabilistic and real-time verification can usually only be performed for small systems because of the state-space explosion problem. It is for this reason that so-called statistical model checking is increasingly being investigated [9]. This technique is similar to simulation in that it executes the system model randomly up to a certain number of runs, but has an important advantage over simulation that properties of interest, including the correctness ones, can still be expressed by logical formulas, since it evaluates them on these runs. As, generally, not all the possible runs are examined, the answer obtained for a formula might be an approximation of the accurate result.

Probabilistic model checking has already been employed for the performance analyses of many kinds of networks and protocols (e.g., [10], [11], [12]), but to our knowledge, it has rarely been used for the validation of analytical models thereof. [13] reports that by using the probabilistic model checker PRISM [14], deficiencies have been found in an analytical web server performance model with proxy cache server. In [15], the results of using PRISM for probabilistic verification of a shuffling protocol for sharing data in a distributed network are compared to the results obtained with an analytical equational model.

PRISM is a powerful tool which supports formal modeling of networks with different kinds of state machines, enables the normal as well as statistical model checking, and can present quantitative results obtained by model checking on graphs [16]. In [17], [18], [19], it is employed for the probabilistic verification of a nonbeacon-enabled version of the IEEE 802.15.4 MAC protocol executed in similar kinds of networks as in [1], but without the aim of using the results to validate an analytical model of the protocol. The purpose of these works has been to devise accurate models of the standard protocol (see, e.g., [20]) and verify some performance properties. [17] and [18] present formal specifications of the networks with two sending nodes by using Markov Decision Processes (MDP) and Probabilistic Timed Automata (PTA) supported by PRISM, but do not provide accurate modeling of the clear-channel assessment period. Besides the improvement of the latter, [19] proposes different approaches to the formal specification of IEEE 802.15.4 star-shaped networks with an arbitrary number of sensor nodes by using PTAs. In [21], the effect of a non-standard clear-channel assessment period length in star-shaped 802.15.4 networks containing hidden nodes is analyzed with PRISM by using statistical model checking.

In contrast to [17], [18], [19], [21], the aim of the research presented in this paper was to try to use PRISM for validation of an analytical model. In particular, the aim was to take the analytical model from [1] as given and to see whether it was possible to use the model checking with PRISM as an alternative to the validation of this model by classical simulation. As is often the case (cf., e.g., [22], [23]), in [1] a MAC protocol slightly different from the standard one is considered, and different performance metrics are dealt with than in [17], [18], [19], [21]. That is why we could not just have used the models and formulas from those papers. The main contribution of this paper are two kinds of network models for PRISM which represent exactly the networks considered in [1], allow us to specify all the performance metrics from the analytical model with probabilistic formulas in PRISM in a simple way, and enable efficient normal, i.e., exhaustive, and, respectively, statistical model checking of them. Note that there is no reason to doubt the claims of [1] about the validity of the analytical model based on the results of simulation. The aim of this paper is, therefore, not to carry out the validation of that model again, but to show how PRISM could be used for its validation advantageously.

In Section 2, we provide a description of the protocol from [1]. In Section 3, we briefly present the mathematical model of the protocol as well as the performance metrics from [1]. In Section 4, we indicate how the considered protocol differs from the standard one and explain both kinds of network models written in PRISM for the former. In Section 5, it is shown how to specify each performance metric in order to be able to use the model checking for it. In Section 6, we present some results of the normal as well as statistical model checking for these metrics. Section 7 contains a discussion and concludes the paper.

Section snippets

MAC protocol

In [1], a nonbeacon-enabled star-shaped wireless personal area network is assumed consisting of a coordinator and N sensor nodes. The analytical model covers the behavior of the nodes after they receive a query from the coordinator. At that moment, “time 0”, each node starts to execute the following unslotted Carrier Sense Multiple Access with Collision Avoidance (CSMA-CA) algorithm in order to send one data frame directly to the coordinator.

The algorithm is based on units of time called

Analytical model

In this subsection, we present the analytical model from [1] to the extent needed to understand the rest of this paper. As in the MAC protocol considered in [1], the backoff delay, the duration of sensing, as well as the duration of transmission are expressed in units of time with duration db, the resolution time of the model is set equal to db, such a unit is called a slot, and the current time is expressed as the current slot number. The jth slot, for j=0,1,2,, means the time from j · db to (

Network specification in PRISM

In PRISM, a system specification consists of modules, which represent concurrent processes [16]. The syntax and semantics of the modules depend on the model type chosen. A difference between the MAC protocol considered in [1] and the standard one considered in [17], [18], [19], [21] is in that the clear channel assessment in the latter does not last a whole backoff period, i.e., 20 symbol times, but only 8 symbol times. If the node assesses the channel to be clear for the whole CCA period, it

Property specification in PRISM

In this section, we show that all the performance metrics listed in Section 3 can be expressed in PRISM’s property specification language. The latter subsumes several probabilistic temporal logics, including PCTL (Probabilistic Computation Tree Logic) [16]. For DTMC models, the queries about probabilities can be specified with formulas of the form P=? [pathprop], where pathprop is an LTL-style (LTL is short for Linear-time Temporal Logic) formula expressing a property of a path in the model. In

Some results of model checking

In this section, we present some results of model-checking the models with signaling with PRISM. In contrast to the simulation, it is possible to obtain exact numerical values for the performance metrics by using the normal model checking in PRISM to the extent allowed by the available computer. We, therefore, first provide some graphs showing the exact results. For comparison, we provide the results obtained with the analytical model on the same graphs.

We implemented the algorithm for the

Conclusions

We showed how PRISM could be applied successfully for the validation of the analytical performance model derived in [1] instead of writing a special simulation program for that purpose. We represented the protocol from [1] in a way that allowed us to query about all the probabilities by referring directly to the slot number, and to obtain the results quickly with normal model checking for small networks and with statistical model checking for larger ones. Instead of adapting an existing PTA or

Acknowledgements

This work was partially funded by the Slovenian Research Agency (ARRS) [contract number P2-0069].

References (25)

  • M. Duflot et al.

    Probabilistic model checking of the CSMA/CD protocol using PRISM and APMC

    Electron. Notes Theor. Comput. Sci.

    (2005)
  • C. Buratti et al.

    Performance analysis of IEEE 802.15.4 non beacon-enabled mode

    IEEE Trans. Veh. Technol.

    (2009)
  • C. Buratti et al.

    Performance analysis of the IEEE 802.15.4 MAC protocol

    Sensor Networks with IEEE 802.15.4 Systems

    (2011)
  • G. Norman, D. Parker, Quantitative verification: Formal guarantees for timeliness, reliability and performance, a...
  • J.-P. Katoen

    Perspectives in probabilistic verification

    Proc. 2nd IFIP/IEEE Int. Symp. Theoretical Aspects of Software Engineering (TASE)

    (2008)
  • M. Kwiatkowska et al.

    Verification of real-time probabilistic systems

  • M. Kwiatkowska et al.

    PRISM: probabilistic model checking for performance and reliability analysis

    Perf. E. R.

    (2009)
  • OMNeT++ Simulation Manual, https://omnetpp.org/doc/omnetpp/manual/ accessed...
  • OPNET: Manual de usuario, Universitat Politècnica de Catalunya, Departament d’Enginyeria Telemàtica, Secció de...
  • A. Legay et al.

    Statistical model checking: An overview

    Proc. 1st Int. Conf. Runtime Verification (RV)

    (2010)
  • M. Kwiatkowska et al.

    Probabilistic model checking of the IEEE 802.11 wireless local area network protocols

    Proc. 2nd Joint Int. Workshop Process Algebra and Probabilistic Methods and Performance Modeling in Verification (PAPMPROBMIV)

    (2002)
  • C. Daws et al.

    Automatic verification of the IEEE 1394 root contention protocol with KRONOS and PRISM

    Int. J. Soft. Tools Technol. Transfer

    (2004)
  • Cited by (0)

    View full text