Skip to main content

Advertisement

Springer Nature Link
Log in

Integration testing of communicating systems with unknown components

  • Published:
annals of telecommunications - annales des télécommunications Aims and scope Submit manuscript

Abstract

The verification of a modular system composed of communicating components is a difficult problem, especially when the formal specifications, i.e. models of the components are not available. Conventional testing techniques are not efficient in detecting erroneous interactions of components because interleavings of internal events are difficult to reproduce in a modular asynchronous system. The problem of detecting intermittent errors and other compositional flaws in the absence of components’ models is addressed in this paper. A method for inferring a controllable approximation of communicating components through testing is elaborated. The inferred finite state models of components are used to detect compositional problems in the system through reachability analysis. To confirm a flaw in a particular component, a witness trace is used to construct a test applied to the component in isolation. The models are refined at each analysis step thus making the approach iterative. The proposed approach employs techniques for machine inference, testing and reachability analysis, and can reveal flaws due to the non-deterministic behaviour of a modular distributed system, such as unspecified receptions, livelocks, divergences, and races that would not be detected by testing alone. One key advantage of the approach is that by means of the inference parameter, it allows one to find a compromise between the co mplexity of testing the integrated system and the precision of the resulting models. The use of large tests is completely avoided in testing a component in isolation, since only single diagnostic test is executed in each iteration. Another advantage of the approach is that inferred models capture the functionalities of components used in the given system; unused behaviours of components are not modelled.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19
Fig. 20
Fig. 21
Fig. 22
Fig. 23

Similar content being viewed by others

Notes

  1. A set of strings is prefix closed if for every string in the set, any of its prefixes are also in the set. Recall that a symbol of an FSM trace is a pair of an input from I and a sequence of outputs from O, so every prefix takes an FSM from its initial state into some state.

  2. For the purpose of composition, each component FSM is unfolded into an IOTS preserving the traces. For details, see the report [40].

  3. This is, of course, not a sufficient, but necessary condition for divergence.

  4. One can use “a single message in transit” as a sufficient condition for their absence [41].

References

  1. Godefroid P (1997) Model checking for programming languages using VeriSoft, In Proceedings of the 24th ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages (POPL). ACM Press, New York, pp 174–186

    Google Scholar 

  2. Clarke EM, Grumberg O, Peled D (1999) Model checking. MIT Press, Cambridge

    Google Scholar 

  3. Kearns MJ, Vazirani UV (1994) An introduction to computational learning theory. MIT Press, Cambridge

    Google Scholar 

  4. Vasilevskii MP (1973) Failure diagnosis of automata. Cybern Syst Anal 9:653–665

    Article  Google Scholar 

  5. Nerode A (1958) Linear automaton transformations. Proc Am Math Soc 9(4):541–544

    Article  MATH  MathSciNet  Google Scholar 

  6. Biermann A, Feldman J (1972) On the synthesis of finite state machines from samples of their behavior. IEEE Trans Comput 21(6):592–597

    Article  MATH  MathSciNet  Google Scholar 

  7. Groz R, Li K, Petrenko A verification of modular systems with unknown components combining testing and inference. Research Report RR-LIG-028. LIG laboratory. http://rr.liglab.fr/research_report/RR-LIG-028.pdf

  8. Trakhtenbrot BA, Barzdin YM (1973) Finite automata, behaviour and synthesis. North-Holland

  9. EU FP7 Project no. 257876, SPaCIoS: Secure Provision and Consumption in the Internet of Services (www.spacios.eu)

  10. Hossen K, Richier J.-L, Oriat C, Groz R (2013) Automatic generation of test drivers for model inference of web applications, in SECTEST 2013 in association with ICST. IEEE

  11. Peled D, Vardi MY, Yannakakis M (1999) Black Box Checking, In Proceedings of 12th IFIP International Conference on Formal Techniques for Networked and Distributed Systems (FORTE), pp 225–240

  12. Balcazar JL, Diaz J, Gavalda R (1997) Algorithms for learning finite automata from queries: a unified view. In Advances in Algorithms, Languages, and Complexity, Kluwer, pp 53–72

  13. de la Higuera C (2010) Grammatical inference: learning automata and grammars, Cambridge University Press

  14. Angluin D (1987) Learning regular sets from queries and counterexamples. Inf Comput 2:87–106

    Article  MathSciNet  Google Scholar 

  15. Howar F, Steffen B, Merten M (2010) From ZULU to RERS, lessons learned in the ZULU challenge. ISoLA, pp 687–704

  16. Rajiv Ranjan S, Rajib M, Srihari S, Manoranjan S (2010) Extracting state models for black-Box software components. J Object Technol 9(3):79–103

    Article  Google Scholar 

  17. Howar F, Isberner M, Merten M, Steffen B (2012) LearnLib tutorial: from Finite Automata to Register Interface Programs. ISoLA, pp 587–590

  18. PetrenkoA, Li K, Groz R, Hossen K, Oriat C (2014) Inferring approximated models for systems engineering, HASE, pp 249–253

  19. Huo J, Petrenko A (2009) Transition covering tests for systems with queues. Softw Test Verification Reliab 19(1):55–83

    Article  Google Scholar 

  20. Tretmans J (1996) Test generation with inputs outputs and repetitive quiescence. Softw Concepts Tools 17(3):103–120

    MATH  Google Scholar 

  21. Lynch N, Tuttle M (1989) An introduction to input/output automata. CWI-Q 2(3):219–246

    MATH  MathSciNet  Google Scholar 

  22. Zafiropulo P, West C, Rudin H, Cowan D, Brand D (1980) Towards analyzing and synthesizing protocols. IEEE Trans Commun 28(4):651–661

    Article  Google Scholar 

  23. Clarke E, Grumberg O, Jha S, Lu Y, Veith H (2000) Counterexample-guided abstraction refinement. In: CAV. Volume 1855 of LNCS, Springer, 154–169

  24. Clarke E, Gupta A, Kukula J, Strichman O (2002) Sat based abstraction-refinement using ilp and machine learning techniques. In: CAV. LNCS, Springer

  25. Grasselli A, Fabrizio L (1965) A method for minimizing the number of internal states in incompletely specified sequential networks. Electron Comput IEEE Trans 3:350–359

    Article  Google Scholar 

  26. Kella J (1971) Sequential machine identification. Comput IEEE Trans 100(3):332–338

    Article  Google Scholar 

  27. Petrenko A, Bochmann G v, Yao M (1996) On fault coverage of tests for finite state specifications. Comput Netw ISDN Syst 29(1):81–106

    Article  Google Scholar 

  28. Oliveira Arlindo L, Stephen A (1995) Edwards. Inference of state machines from examples of behavior, UCB/ERL Tech. Rep. M 95

  29. Holzmann GJ (1997) The model checker SPIN. IEEE Trans Software Eng 23(5):279–295

    Article  MathSciNet  Google Scholar 

  30. Andrews T, Qadeer S, Rajamani SK, Rehof J, Xie ZingY A model checker for concurrent software, MSR Technical Report: MSR-TR-2004-10

  31. Elkind E, Genest B, Peled D, QuH (2006) Grey-box checking. In Proceedings of 26th IFIP International Conference on Formal Techniques for Networked and Distributed Systems (FORTE), pp 420–435

  32. Cook JE, Wolf AL (1998) Discovering models of software processes from event-based data. ACM Trans Softw Eng Methodol 7(3):215–249

    Article  Google Scholar 

  33. Pasareanu CS, Giannakopoulou D, Bobaru MG, Cobleigh JM, Barringer H (2008) Learning to divide and conquer: applying the L* algorithm to automate assume-guarantee reasoning. Form Methods Syst Des 32(3):175–205

    Article  MATH  Google Scholar 

  34. Hallal HH, Boroday S, Petrenko A, Ulrich A (2006) A formal approach to testing properties in causally consistent distributed traces. Form Asp Comput 18(1):63–83

    Article  MATH  Google Scholar 

  35. Mariani L, Pezzè M (2005) Behavior capture and test: automated analysis of component integration. In Proceedings of 10th International Conference on Engineering of Complex Computer Systems (ICECCS), pp 292–301

  36. Bertolino A, Muccini H, Polini A (2006) Architectural verification of black-box component-based systems. In Proceedings of the 3rd International Workshop on Rapid Integration of Software Engineering Techniques (RISE), LNCS 4401, pp 98–113

  37. Li K, Groz R, Shahbaz M (2006) Integration testing of components guided by incremental state machine learning. In Proceedings of Testing: Academia and Industry Conference—Practice and Research Techniques (TAIC PART), pp 231–247

  38. Shahbaz M, Li K, Groz R (2007) learning and integration of parameterized components through testing. In Proceedings of 19th IFIP International Conference on Testing of Software and Communication Systems (TestCom), pages 319–334

  39. Shahbaz M, Groz R (2013) Analysis and testing of black-box component-based systems by inferring partial models. J Softw Test Verification Reliab. doi:10.1002/stvr.1491

    Google Scholar 

  40. Groz R, Li K, Petrenko A, Shahbaz M (2008) modular system verification by inference, testing and reachability analysis, In Proceedings of 20th IFIP International Conference on Testing of Software and Communication Systems (TestCom), pp. 216–233

  41. Petrenko A, Yevtushenko N (1998) Solving asynchronous equations. In Proceedings of 11th IFIP International Conference on Formal Techniques for Networked and Distributed Systems (FORTE), pp 231–247

Download references

Acknowledgments

This work was partially supported by the EU FP7 Project no. 257876, “SPaCIoS: Secure Provision and Consumption in the Internet of Services” (www.spacios.eu) [10]. The authors acknowledge the work of Laurent Anadon who provided the performance measurements for SIMPA and LearnLib, as reported in Section 3.3.

Author information

Authors and Affiliations

  1. Université Grenoble Alpes, LIG, Grenoble, France

    Roland Groz

  2. SAP Product Security Research, Sophia Antipolis, Grenoble, France

    Keqin Li

  3. Computer Research Institute of Montreal (CRIM), Montreal, Quebec, Canada

    Alexandre Petrenko

Authors
  1. Roland Groz
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Keqin Li
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Alexandre Petrenko
    View author publications

    You can also search for this author inPubMed Google Scholar

Corresponding author

Correspondence to Alexandre Petrenko.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Groz, R., Li, K. & Petrenko, A. Integration testing of communicating systems with unknown components. Ann. Telecommun. 70, 107–125 (2015). https://doi.org/10.1007/s12243-014-0449-0

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12243-014-0449-0

Keywords