Passive testing and application to the GSM-MAP protocol

doi:10.1016/S0950-5849(99)00039-7

Information and Software Technology

Volume 41, Issues 11–12, 15 September 1999, Pages 813-821

https://doi.org/10.1016/S0950-5849(99)00039-7 Get rights and content

Abstract

Passive testing is the process of collecting traces of messages exchanged between an operating implementation and its environment, in order to verify that these traces actually belong to the language accepted by the provided finite state machine specification. In this paper, we present an extension of the existing algorithms to consider an extended finite state machine as the specification. An algorithm is also introduced to take into account the number of transitions covered. These techniques are illustrated by the application to a real protocol, the GSM (global system for mobile communication)-MAP (mobile application part).

Introduction

Passive testing is the process of collecting traces of messages exchanged between an operating implementation under test (IUT) and its environment, and verifying whether these traces actually belong to the language accepted by the specification automaton.

Though passive testing is sometimes mentioned as an alternative to active testing [1], only little effort has been devoted to this aspect of testing [2], [3]. However, we feel that passive testing is worth investigating, since (a) under certain circumstances, it may be the only type of test available, e.g. in network management, (b) it is relatively cheap and easy to implement, and (c) active testing is sometimes impractical due to the complexity of systems. In this paper, we present the principles of passive testing, and an application to a real protocol, the GSM (global system for mobile communication)-MAP (mobile application part). We also extend the algorithm proposed in Ref. [3] in order to take the transition coverage into account and in order to consider an extended finite state machine as the system specification.

The paper is organized as follows. In Section 2, we present the concepts of passive testing for finite state machines. Section 3 presents two algorithms to evaluate test coverage, taking transition coverage into account. In Section 4, we study the application of this method to real protocols, and we introduce an extension of the algorithm in order to consider extended finite state machines as specifications. Section 5 is an application of this method to the GSM-MAP-DSM protocol.

Section snippets

The basics of passive testing

In passive testing [3], contrary to active testing, the tester does not control the implementation under test. The implementation is in an operating condition, and the tester only observes the messages exchanged by the IUT and its environment, in order to check if they correspond to a behaviour compliant with the specification (see Fig. 1).

Related work are trace analyses [4], oracle generation [5] and fault diagnostic [6]. A major difference with passive testing is that, in these approaches, it

Test coverage

One of the problems with passive testing is that it is impossible to issue a PASS verdict. Contrary to active testing, we cannot control the implementation under test in order to cover all the transitions. Therefore, it may possibly happen that infrequent parts of the system's behaviour remain untested. Nevertheless, we can try to determine the test coverage, i.e compute the number of transitions that have been fired, and indicate which transitions have been fired and which have not.

Extended finite state machine

Real life protocols are too complex to be modelled with a simple Finite State Machine. In order to specify a real protocol, we must use the Extended Finite State Machine (EFSM) formalism (i.e. there will be internal variables and parameters). There may be a predicate on variables associated with each transition of EFSM. The predicate must be true for the transition to be fireable. A transition may also have actions on the variables, modifying their values. In this paper, we consider only

An experiment with the GSM-MAP protocol

The example we propose here is the passive testing of the GSM-MAP extended finite state machine specification, for which we have used the first approach (working on the reachability graph) to test the GSM-MAP protocol. Using this GSM-MAP extended finite state machine, we observe a trace of the execution of the protocol. In this section, we provide a short description of the protocol and we describe the results we have obtained.

Conclusion and future work

In this paper, we have applied passive testing techniques to a real protocol, the GSM-MAP. The experiments we performed showed that passive testing is a promising technique. It can be applied to test systems where active testing is hard to perform or where no control is possible over the system to be tested. It can also be very useful to improve the specification used for the observation. In fact, some of the discrepancies we found during the experiments were produced by ambiguities in the

Acknowledgements

We are indebted to David Lee, who helped us with very useful discussions on this subject and to Melania Ionescu who helped us with the GSM-MAP SDL specification.

References (19)

RM-ODP, Open distributed Processing—Reference Model—Part 2: Foundations, International Standard 10746-2/ITU-T...
C.L. Seitz
An approach to designing checking experiments based on a dynamic model
D. Lee et al., Passive testing and applications to network management, in: ICNP’97 International Conference on Network...
G.v. Bochmann et al.
Trace analysis for conformance and arbitration testing
IEEE Transactions on Software Engineering
(1989)
R. Dssouli, K. Saleh, E. Aboulhamid, A. En-Nouaary, C. Bourhfir, Test development for communication protocols: towards...
A. Ghedamsi, R. Dssouli, G.v. Bochmann, Diagnostic tests for single transition faults in non-deterministic finite state...
B. Sarikaya et al.
Obtaining normal form specifications for protocols
(1986)
H. Ural et al.
A test sequence selection method for protocol testing
IEEE Transactions on Communications
(1991)
C.-M. Huang, Y.-C. Lin M.-Y. Jang, An executable protocol test sequence generation method for EFSM-specified protocols,...

There are more references available in the full text version of this article.

Cited by (28)

A tool supported methodology to passively test asynchronous systems with multiple users
2018, Information and Software Technology
Citation Excerpt :
In contrast, there has been some work on active testing and several approaches have appeared in the literature for models where there is a distinction between inputs and outputs [15,16,21,37,45,52]. Homing algorithms are used in many approaches to passive testing, in particular in the seminar contributions of the 1990s [24,46], there exist surveys on homing algorithms [40] and this is still an active research area [49]. A homing algorithm applies a sequence of actions to a system so that it brings it to its initial state (or to a certain special state where we know that we can start testing the SUT).
Context: Testing usually involves the interaction of the tester with the system under test. However, there are many situations in which this interaction is not feasible and so one requires a passive approach in which the system is analysed to look for failures or unexpected behaviours. The entities of a complex system usually communicate in an asynchronous manner and this complicates the testing tasks since the observed order of events need not be the same as the order in which the events were produced. In previous work, we presented a formal passive testing theory for a single user and system communicating through an asynchronous channel. We were able to check that a trace generated by the system satisfies a given property.
Objective: This papers extends our work, for detecting potential intrusions and unexpected behaviours, to the case where many users simultaneously communicate with a central server. We evaluate the practical value of the theoretical framework with a non-trivial system.
Method: We developed a novel complete theoretical framework to analyse asynchronous systems with multiple users. All the algorithms are fully implemented. Experiments were performed over the Nextcloud platform to show the applicability of our framework. The experiments include an attack so that actual vulnerabilities could be revealed.
Results: The application of our methodology, supported by a tool fully implementing it, was able to reveal a vulnerability in the WebDAV protocol running on Nextcloud.
Conclusion: The extension of our previous work is not only useful from a theoretical point of view but also increases the applicability of the original work. We are now able to analyse systems where the interaction with different users can lead to unexpected behaviours. We have been able to find a vulnerability in a real system, showing the usefulness of our work.
Using passive testing based on symbolic execution and slicing techniques: Application to the validation of communication protocols
2013, Computer Networks
Citation Excerpt :
These messages are defined as control and data portions based on the function of the protocol. Many works on passive testing [6–9] are focused only on checking the control portion of the protocol without taking into account the data part. However, it may result in producing false positive verdicts as illustrated below with an example.
This paper presents a new approach to perform passive testing based on the analysis of the control and data part of the system under test. Passive testing techniques are based on the observation and verification of properties on the behaviour of a system without interfering with its normal operation. Many passive testing techniques consider only the control part of the system and neglect data, or are confronted with an overwhelming amount of data values to process. In our approach, we consider control and data parts by integrating the concepts of symbolic execution and we improve trace analysis by introducing trace slicing techniques. Properties are described using Input–Output Symbolic Transition Systems (IOSTSs) and we illustrate in the paper how they can be tested on real execution traces optimizing the trace analysis. These properties can be designed to test the functional conformance of a protocol as well as security properties.
In addition to the theoretical approach, we have developed a software tool that implements the algorithms presented in this paper. Finally, as a proof of concept of our approach and tool we have applied the techniques to a real-life case study: the SIP protocol. In particular, the proposed techniques are applied to a set of real execution traces extracted from an IMS/SIP architecture.
An advanced approach for modeling and detecting software vulnerabilities
2012, Information and Software Technology
Citation Excerpt :
In this way it is possible to compare the execution traces to the FSM in order to detect faults in the implementation. Passive testing has been used in many different contexts, e.g. on an FSM model of a network, in network management to detect configuration provisioning and in a GSM-MAP protocol [26]. TestInv-Code, developed by Montimage, is a prototype passive testing tool that accepts formal vulnerability models written using VDCs.
Passive testing is a technique in which traces collected from the execution of a system under test are examined for evidence of flaws in the system.
In this paper we present a method for detecting the presence of security vulnerabilities by detecting evidence of their causes in execution traces. This is a new approach to security vulnerability detection.
Our method uses formal models of vulnerability causes, known as security goal models and vulnerability detection conditions (VDCs). The former are used to identify the causes of vulnerabilities and model their dependencies, and the latter to give a formal interpretation that is suitable for vulnerability detection using passive testing techniques. We have implemented modeling tools for security goal models and vulnerability detection conditions, as well as TestInv-Code, a tool that checks execution traces of compiled programs for evidence of VDCs.
We present the full definitions of security goal models and vulnerability detection conditions, as well as structured methods for creating both. We describe the design and implementation of TestInv-Code. Finally we show results obtained from running TestInv-Code to detect typical vulnerabilities in several open source projects. By testing versions with known vulnerabilities, we can quantify the effectiveness of the approach.
Although the current implementation has some limitations, passive testing for vulnerability detection works well, and using models as the basis for testing ensures that users of the testing tool can easily extend it to handle new vulnerabilities.
A passive testing approach based on invariants: Application to the WAP
2005, Computer Networks
This paper presents a new methodology to perform passive testing based on invariants. This novel approach is supported by the following idea: a set of invariants represent the most relevant expected properties of the implementation under test. Intuitively, an invariant expresses the fact that each time the implementation under test performs a given sequence of actions, then it must exhibit a behavior reflected in the invariant. For example, an invariant such as i₁/o₁, …, i_n−1/o_n−1, i_n/O must be interpreted as “each time the implementation performs the sequence i₁/o₁, …, i_n−1/o_n−1, i_n the next observed output belongs to the set O”. We call these invariants simple invariants. In this work we introduce a new notion of invariants to deal with more subtle properties. For instance, we will consider invariants to express properties such as “if y happens then we must have that x had happened before”. These invariants are called obligation invariants. We present algorithms to decide the correctness of the proposed invariants with respect to a given specification. Once we have that an invariant is correct with respect to a given specification, we check whether the execution traces observed from the implementation respect the invariant. In order to perform this phase we present two algorithms based, respectively, on left-to-right and right-to-left pattern matching algorithms.
In addition to the theoretical framework we have developed a software tool, called TestInv, that helps in the automation of our passive testing approach. In particular, the algorithms presented in this paper are fully implemented in the tool. Finally, in order to test the usefulness of our approach we have chosen a real-life case study: the Wireless Application Protocol (WAP). We present a test architecture as well as the most relevant results obtained from the application of our approach to the WAP.
New approaches for passive testing using an Extended Finite State Machine specification
2003, Information and Software Technology
Citation Excerpt :
We wanted to compare the effectiveness of our methods with these types of approach to show the contributions brought by our work. Results proved to be positive since the experiments showed that certain errors detected by our methods were undetected by others (such as Ref. [2]). This work has many prospects.
This paper presents two new approaches for passive testing using an Extended Finite State Machine (EFSM) specification. The state of the art of passive testing shows us that all the methods for detection of errors based on EFSMs try to match the trace to the specification. Indeed, one searches a state succession in the specification machine that is able to generate the trace observed on the implementation. Using this approach, processing is performed on the specification and the trace remains in the background since no operation is applied to it. This made us realise that focusing our efforts on the trace could be beneficial and has given as result two approaches presented in this paper that extract information from the specification and then work on the trace. Thus, they take a different direction than the previous methods. We first present an approach to test traces by using invariants resulting from the specification. We formally define these invariants and we see how to extract them. We also discuss their ability to detect errors appearing in the implementation. This approach is able to test the data flow, but not in a very satisfactory way. This is the reason for a second approach seeking to apply a set of constraints to the trace. We develop in detail its principles. Both approaches are applied to a Simple Connection Protocol (SCP) and the results of preliminary experiments are presented.
A survey on formal active and passive testing with applications to the cloud
2015, Annales des Telecommunications/Annals of Telecommunications

View all citing articles on Scopus

View full text

Passive testing and application to the GSM-MAP protocol

Abstract

Introduction

Section snippets

The basics of passive testing

Test coverage

Extended finite state machine

An experiment with the GSM-MAP protocol

Conclusion and future work

Acknowledgements

An approach to designing checking experiments based on a dynamic model

Trace analysis for conformance and arbitration testing

IEEE Transactions on Software Engineering

Obtaining normal form specifications for protocols

A test sequence selection method for protocol testing

IEEE Transactions on Communications