Skip to main content
SpringerLink
Log in

Analyzing program behavior through active automata learning

  • Rers
  • Published:
International Journal on Software Tools for Technology Transfer Aims and scope Submit manuscript

Abstract

The objective of the RERS Challenge 2013 was to analyze program behavior with respect to given sets of LTL and reachability properties for a set of reactive programs. The programs in various sizes and complexities could be divided into three different categories, depending on the available information: from black-box (binary-only) to white-box (full source code) over a mixed form thereof (grey-box). In this paper we present our approach to tackling the challenge problems, which is based on active automata learning. This required extending automata learning algorithms to exploit the given information, and adapting them in order to overcome problem-specific obstacles. We describe general optimizations and discuss the achieved results.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

Notes

  1. http://www.learnlib.de/.

  2. Even though quiescence by every right can be considered normal behavior of a reactive system, it poses a special difficulty for automata learning which requires observable behavior.

  3. Indeed, already all continuations of \(5\,3\,3\,2\) will result in an errorCheck in the output such that the lower part of the table does not only contain at least \(6\), it already contains at least \(36\) saved queries for this access sequence.

  4. Of course this only works if the output is limited to a maximum of one line per input, which was the case in the challenge problems.

  5. Please note that we use the \(W\)-method on a hypothesis automaton with over one thousand states and an alphabet of size 20.

References

  1. Aarts, F., Schmaltz, J., Vaandrager, F.: Inference and abstraction of the biometric passport. In proceedings of the 4th international conference on leveraging applications of formal methods, verification, and validation—volume part I, ISoLA’10, pp. 673–686, Springer, Berlin (2010)

  2. Aarts, F., Vaandrager, F.: Learning i/o automata. In proceedings of the 21st international conference on concurrency theory, CONCUR’10, pp. 71–85. Springer, Berlin, (2010)

  3. Angluin, Dana: Learning regular sets from queries and counterexamples. Inf. Comput. 75(2), 87–106 (1987)

    Article  MATH  MathSciNet  Google Scholar 

  4. Oliver, B., Neubauer, J., Steffen, B., Howar F.: Reusing system states by active learning algorithms. In Alessandro Moschitti and Riccardo Scandariato, (eds.) Eternal systems, communications in computer and information science, vol. 255, pp. 61–78. Springer, Berlin, (2012)

  5. Beyer, D., Stahlbauer A.: BDD-based software verification. Applications to event-condition-action systems. Software tools for technology transfer. doi:10.1007/s10009-014-0334-1 (2014)

  6. Cho, C.Y., Babić, D., Poosankam, P., Chen, K.Z., Wu, E.X., Song, D.: MACE: model-inference-assisted concolic exploration for protocol and vulnerability discovery. In proceedings of the 20th USENIX security symposium (2011)

  7. Cho, C.Y., Babić, D., Shin, R., Song, D.: Inference and analysis of formal models of botnet command and control protocols. In CCS’10: proceedings of the 2010 ACM conference on computer and communications security, pp. 426–440. ACM (2010)

  8. Chow, Tsun S.: Testing software design modeled by finite-state machines. IEEE Trans. Softw. Eng. 4(3), 178–187 (May 1978)

  9. Clarke, Edmund, Biere, Armin, Raimi, Richard, Zhu, Yunshan: Bounded model checking using satisfiability solving. Form. Methods Syst. Des 19(1), 7–34 (2001)

    Article  MATH  Google Scholar 

  10. Clarke, E.M. Jr., Grumberg, O., Peled D.A.: Model Checking. MIT Press, Cambridge, MA, USA (1999)

  11. Emerson, E.Allen, Halpern, Joseph Y.: Decision procedures and expressiveness in the temporal logic of branching time. J. Comput. Syst. Sci. 30(1), 1–24 (1985)

    Article  MATH  MathSciNet  Google Scholar 

  12. Gastin, P., Oddoux, D.: Fast ltl to büchi automata translation. In Berry G., Comon H., Finkel A., (eds.) Computer aided verification, lecture notes in computer science, vol. 2102 pp. 53–65. Springer, Berlin (2001)

  13. Giannakopoulou, D., Lerda, F.: From states to transitions: improving translation of ltl formulae to buchi automata. In Proceedings FORTE’02., LNCS, vol. 2529, pp. 308–326. Springer (2002)

  14. Hagerer, A., Hungar, H., Niese, O., Steffen, B.: Model generation by moderated regular extrapolation. In Kutsche R.D., Weber H. (eds.) FASE, Lecture notes in computer science, vol. 2306, pp. 80–95. Springer (2002)

  15. Howar, F., Isberner, M., Merten, M., Steffen, B., Beyer, D.: The rers grey-box challenge 2012: analysis of event-condition-action systems. In Margaria T., Steffen B. (eds.) ISoLA (1), lecture notes in computer science, vol. 7609, pp. 608–614. Springer (2012)

  16. Howar, F., Isberner, M., Merten, M., Steffen, B., Beyer, D., Corina S.: Păsăreanu. Rigorous examination of reactive systems. The RERS challenges 2012 and 2013. Software tools for technology transfer. doi:10.1007/s10009-014-0337-y (2014)

  17. Howar, F., Steffen, B., Merten, M.: Automata learning with automated alphabet abstraction refinement. In proceedings of the 12th international conference on verification, model checking, and abstract interpretation, VMCAI’11, pp. 263–277. Springer, Berlin (2011)

  18. Hungar, H., Niese, O., Steffen, B.: Domain-specific optimization in automata learning. In computer aided verification, vol. 2725 LNCS, pp. 315–327. Springer (2003)

  19. Irfan, M.N., Oriat, C., Groz, R.: Angluin style finite state machine inference with non-optimal counterexamples. In proceedings of the first international workshop on model inference in testing, MIIT 10, pp. 11–19. New York, USA, ACM (2010)

  20. Isberner, M., Howar, F., Steffen, B.: Inferring automata with state-local alphabet abstractions. In Brat G., Rungta N., Venet A., (eds.) NASA formal methods, Lecture Notes in Computer Science, vol. 7871, pp. 124–138. Springer, Berlin (2013)

  21. Isberner, M., Howar, F., Steffen, B.: Learning register automata: from languages to program structures. Machine Learning, pp. 1–34. (2013)

  22. King, J.C.: Symbolic Execution and Program Testing. Commun. ACM 19(7), 385–394 (July 1976)

  23. Kroening, D., Strichman, O.: Decision procedures: an algorithmic point of view. Springer Publishing Company, Incorporated, 1 edition (2008)

  24. Maler, O., Mens, I.: Learning regular languages over large alphabets. In Ábrahám E., Havelund K. (eds.) Tools and algorithms for the construction and analysis of systems, Lecture notes in computer science, vol. 8413, pp. 485–499. Springer, Berlin (2014)

  25. Margaria, T., Niese, O., Raffelt, H., Steffen, B.: Efficient test-based model generation for legacy reactive systems. In HLDVT ’04, pp. 95–100, Washington, DC, USA, IEEE computer society (2004)

  26. McMillan, K.L.: Symbolic model checking: an approach to the state explosion problem. PhD thesis, Carnegie Mellon University, Pittsburgh, PA, USA, UMI Order No. GAX92-24209 (1992)

  27. Merten, M., Steffen, B., Howar, F., Margaria, T.: Next generation learnlib. In proceedings of the 17th international conference on tools and algorithms for the construction and analysis of systems: part of the joint European conferences on theory and practice of software, TACAS’11/ETAPS’11, pp. 220–223. Springer, Berlin (2011)

  28. Morse, J., Cordeiro, L., Nicole, D., Fischer, B.: Applying symbolic bounded model checking to the 2012 RERS greybox challenge. Software tools for technology transfer. doi:10.1007/s10009-014-0335-0 (2014)

  29. Müller-Olm, M., Schmidt, D.A., Steffen, B.: Model-checking: A tutorial introduction. In proceedings of the 6th international symposium on static analysis, SAS ’99, pp. 330–354. Springer, London (1999)

  30. Nerode, A.: Linear automaton transformations. Proc. Am. Math. Soc. 9(4), 541–544 (1958)

    Article  MATH  MathSciNet  Google Scholar 

  31. Niese, O.: An integrated approach to testing complex systems. PhD thesis, University of Dortmund, Germany (2003)

  32. Peled, Doron, Vardi, Moshe Y., Yannakakis, Mihalis: Black box checking. J. Autom. Lang. Comb. 7(2), 225–246 (2001)

    MathSciNet  Google Scholar 

  33. Pnueli, A.: The temporal logic of programs. In FOCS, IEEE Comput. Soc., pp. 46–57 (1977)

  34. Raffelt, H., Steffen, B., Berg, T., Margaria, T.: Learnlib: a framework for extrapolating behavioral models. International Journal on Software Tools for Technology Transfer 11(5), 393–407 (2009)

    Article  Google Scholar 

  35. Rivest, Ronald L., Schapire, Robert E.: Inference of finite automata using homing sequences. Inf. Comput. 103(2), 299–347 (1993)

    Article  MATH  MathSciNet  Google Scholar 

  36. Schordan, M., Prantl, A.: Combining static analysis and state transition graphs for verification of event-condition-action systems in the rers 2012 and 2013 challenges. Software Tools for Technology Transfer. doi:10.1007/s10009-014-0338-x (2014)

  37. Shahbaz, M., Groz, R.: Inferring mealy machines. In proceedings of the 2nd world congress on formal methods, FM ’09, pp. 207–222. Springer, Berlin (2009)

  38. Steffen, B., Howar, F., Isberner, M., Naujokat, S., Margaria, T.: Tailored generation of concurrent benchmarks. Software Tools for Technology Transfer, this volume (2014)

  39. Steffen, B., Howar, F., Merten, M.: Introduction to active automata learning from a practical perspective. In Bernardo M., Issarny V. (eds.) Formal methods for eternal networked software systems, Lecture notes in computer science, vol. 6659, pp. 256–296. Springer, Berlin (2011)

  40. Steffen, B., Isberner, M., Naujokat, S., Margaria, T., Geske, M.: Property-driven benchmark generation. In Bartocci E., Ramakrishnan C.R. (eds.) Model checking software, Lecture notes in computer science, vol. 7976, pp. 341–357. Springer, Berlin (2013)

  41. Steffen, B., Isberner, M., Naujokat, S., Margaria, T., Geske, M.: Property-driven benchmark generation: synthesizing programs of realistic structure. Software tools for technology transfer. doi:10.1007/s10009-014-0336-z (2014)

  42. van de Pol, J.C., Ruys, T.C., Brinke, S.: Thoughtful brute force attack of the RERS 2012 and 2013 challenges. Software tools for technology transfer. doi:10.1007/s10009-014-0324-3 (2014)

  43. Vardi, M.Y.: An automata-theoretic approach to linear temporal logic. In proceedings of the VIII Banff higher order workshop conference on logics for concurrency: structure versus automata: structure versus automata, pp. 238–266. Secaucus, NJ, USA, Springer, NewYork Inc (1996)

Download references

Author information

Authors and Affiliations

  1. Dortmund University of Technology, 44227, Dortmund, Germany

    Oliver Bauer, Maren Geske & Malte Isberner

Authors
  1. Oliver Bauer
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Maren Geske
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Malte Isberner
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Oliver Bauer.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Bauer, O., Geske, M. & Isberner, M. Analyzing program behavior through active automata learning. Int J Softw Tools Technol Transfer 16, 531–542 (2014). https://doi.org/10.1007/s10009-014-0333-2

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10009-014-0333-2

Keywords

Search

Navigation