Skip to main content
SpringerLink
Log in

Model-Based Testing Without Models: The TodoMVC Case Study

  • Chapter
  • First Online:
ModelEd, TestEd, TrustEd

Part of the book series: Lecture Notes in Computer Science ((LNPSE,volume 10500))

Abstract

Web applications define the interface to many of the businesses and services that we interact with and use on a daily basis. The technology stack enabling these applications is constantly changing and applications are accessed from a plethora of different devices. Automated testing of the behavior of applications is a promising strategy for reducing the manual effort that has to be spent on ensuring a consistent user experience across devices. Unfortunately, specifications or models of the desired behavior often do not exist. Model-based testing without models (aka learning-based testing) tries to overcome this hurdle by integrating model learning and model-based testing. In this paper, we sketch the ALEX tool [1, 11] for learning-based testing of web application and demonstrate its operation on benchmarks from the TodoMVC project. Our learning-based conformance analysis reveals that 7 of 27 Todo-apps exhibit behavior that differs from the majority of implementations.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    http://learnlib.de/.

  2. 2.

    http://learnlib.github.io/alex/.

  3. 3.

    http://www.seleniumhq.org/.

  4. 4.

    http://todomvc.com/.

  5. 5.

    This has been enforced by adding code to the implementations of test inputs that checks whether a task is absent or created, but currently not visible — similar to a test purpose.

  6. 6.

    https://sites.google.com/a/chromium.org/chromedriver/.

  7. 7.

    http://learnlib.github.io/alex/book/1.2.1/contents/examples/todomvc/index.html.

References

  1. Automata learning experience (2016). http://learnlib.github.io/alex/. Accessed 23 Oct 2016

  2. Todomvc (2016). http://todomvc.com/. Accessed 23 Oct 2016

  3. Todomvc specification (2016). https://github.com/tastejs/todomvc/blob/master/app-spec.md. Accessed 21 Oct 2016

  4. Aarts, F., Jonsson, B., Uijen, J.: Generating models of infinite-state communication protocols using regular inference with abstraction. In: Petrenko, A., Simão, A., Maldonado, J.C. (eds.) ICTSS 2010. LNCS, vol. 6435, pp. 188–204. Springer, Heidelberg (2010). doi:10.1007/978-3-642-16573-3_14

    Chapter  Google Scholar 

  5. Aarts, F., Kuppens, H., Tretmans, J., Vaandrager, F., Verwer, S.: Improving active mealy machine learning for protocol conformance testing. Mach. Learn. 96(1), 189–224 (2014)

    Article  MathSciNet  MATH  Google Scholar 

  6. Aarts, F., Kuppens, H., Tretmans, J., Vaandrager, F.W., Verwer, S.: Learning and testing the bounded retransmission protocol. In: Proceedings of the Eleventh International Conference on Grammatical Inference, ICGI 2012. University of Maryland, College Park, USA, 5–8 September 2012, pp. 4–18 (2012)

    Google Scholar 

  7. Alur, R., Madhusudan, P.: Visibly pushdown languages. In: Proceedings of the Thirty-sixth Annual ACM Symposium on Theory of Computing, STOC 2004, pp. 202–211. ACM, New York, NY, USA (2004)

    Google Scholar 

  8. Angluin, D.: Learning regular sets from queries and counter examples. Inf. Comput. 75(2), 87–106 (1987)

    Article  MATH  Google Scholar 

  9. Argyros, G., Stais, I., Kiayias, A., Keromytis, A.D.: Back in black: towards formal, black box analysis of sanitizers and filters. In: 2016 IEEE Symposium on Security and Privacy (SP), pp. 91–109, May 2016

    Google Scholar 

  10. Avasarala, S.: Selenium WebDriver Practical Guide. Packt Publishing, Birmingham (2014). ISBN 9781782168850

    Google Scholar 

  11. Bainczyk, A., Schieweck, A., Isberner, M., Margaria, T., Neubauer, J., Steffen, B.: ALEX: mixed-mode learning of web applications at ease. In: Margaria, T., Steffen, B. (eds.) ISoLA 2016. LNCS, vol. 9953, pp. 655–671. Springer, Cham (2016). doi:10.1007/978-3-319-47169-3_51

    Chapter  Google Scholar 

  12. Belinfante, A.: JTorX: a tool for on-line model-driven test derivation and execution. In: Esparza, J., Majumdar, R. (eds.) TACAS 2010. LNCS, vol. 6015, pp. 266–270. Springer, Heidelberg (2010). doi:10.1007/978-3-642-12002-2_21

    Chapter  Google Scholar 

  13. Berg, T., Grinchtein, O., Jonsson, B., Leucker, M., Raffelt, H., Steffen, B.: On the correspondence between conformance testing and regular inference. In: Cerioli, M. (ed.) FASE 2005. LNCS, vol. 3442, pp. 175–189. Springer, Heidelberg (2005). doi:10.1007/978-3-540-31984-9_14

    Chapter  Google Scholar 

  14. Bertolino, A., Calabrò, A., Merten, M., Steffen, B.: Never-stop learning: continuous validation of learned models for evolving systems through monitoring. ERCIM News 2012(88), 28–29 (2012)

    Google Scholar 

  15. Brinksma, E., Tretmans, J.: Testing transition systems: an annotated bibliography. In: Cassez, F., Jard, C., Rozoy, B., Ryan, M.D. (eds.) MOVEP 2000. LNCS, vol. 2067, pp. 187–195. Springer, Heidelberg (2001). doi:10.1007/3-540-45510-8_9

    Chapter  Google Scholar 

  16. Cassel, S., Howar, F., Jonsson, B., Steffen, B.: Active learning for extended finite state machines. Formal Aspects Comput. 28(2), 233–263 (2016)

    Article  MathSciNet  MATH  Google Scholar 

  17. Choi, W., Necula, G., Sen, K.: Guided GUI testing of android apps with minimal restart and approximate learning. SIGPLAN Not. 48(10), 623–640 (2013)

    Article  Google Scholar 

  18. Chow, T.S.: Testing software design modeled by finite-state machines. IEEE Trans. Softw. Eng. SE–4(3), 178–187 (1978)

    Article  MATH  Google Scholar 

  19. Dinca, I., Ipate, F., Mierla, L., Stefanescu, A.: Learn and test for event-B – A rodin plugin. In: Derrick, J., Fitzgerald, J., Gnesi, S., Khurshid, S., Leuschel, M., Reeves, S., Riccobene, E. (eds.) ABZ 2012. LNCS, vol. 7316, pp. 361–364. Springer, Heidelberg (2012). doi:10.1007/978-3-642-30885-7_32

    Chapter  Google Scholar 

  20. Drews, S., D’Antoni, L.: Learning symbolic automata. In: Legay, A., Margaria, T. (eds.) TACAS 2017. LNCS, vol. 10205, pp. 173–189. Springer, Heidelberg (2017). doi:10.1007/978-3-662-54577-5_10

    Chapter  Google Scholar 

  21. Flanagan, D.: JavaScript: The Definitive Guide: Activate Your Web Pages (Definitive Guides), 6th edn. O’Reilly Media, Sebastopol (2011)

    Google Scholar 

  22. Fujiwara, S., von Bochmann, G., Khendek, F., Amalou, M., Ghedamsi, A.: Test selection based on finite state models. IEEE Trans. Softw. Eng. 17(6), 591–603 (1991)

    Article  Google Scholar 

  23. Hagerer, A., Hungar, H., Niese, O., Steffen, B.: Model generation by moderated regular extrapolation. In: Kutsche, R.-D., Weber, H. (eds.) FASE 2002. LNCS, vol. 2306, pp. 80–95. Springer, Heidelberg (2002). doi:10.1007/3-540-45923-5_6

    Chapter  Google Scholar 

  24. Hagerer, A., Margaria, T., Niese, O., Steffen, B., Brune, G., Ide, H.-D.: Efficient regression testing of CTI-systems: testing a complex call-center solution. Ann. Rev. Commun. Int. Eng. Consortium (IEC) 55, 1033–1040 (2001)

    Google Scholar 

  25. Howar, F., Steffen, B., Merten, M.: Automata learning with automated alphabet abstraction refinement. In: Jhala, R., Schmidt, D. (eds.) VMCAI 2011. LNCS, vol. 6538, pp. 263–277. Springer, Heidelberg (2011). doi:10.1007/978-3-642-18275-4_19

    Chapter  Google Scholar 

  26. Isberner, M.: Foundations of active automata learning: an algorithmic perspective. Ph.D. thesis. TU Dortmund University, October 2015

    Google Scholar 

  27. Isberner, M., Howar, F., Steffen, B.: Inferring automata with state-local alphabet abstractions. In: Brat, G., Rungta, N., Venet, A. (eds.) NFM 2013. LNCS, vol. 7871, pp. 124–138. Springer, Heidelberg (2013). doi:10.1007/978-3-642-38088-4_9

    Chapter  Google Scholar 

  28. Isberner, M., Howar, F., Steffen, B.: Learning register automata: from languages to program structures. Mach. Learn. 96, 1–34 (2013)

    MathSciNet  MATH  Google Scholar 

  29. Isberner, M., Howar, F., Steffen, B.: The TTT algorithm: a redundancy-free approach to active automata learning. In: Bonakdarpour, B., Smolka, S.A. (eds.) RV 2014. LNCS, vol. 8734, pp. 307–322. Springer, Cham (2014). doi:10.1007/978-3-319-11164-3_26

    Google Scholar 

  30. Isberner, M., Howar, F., Steffen, B.: The open-source learnlib. In: Kroening, D., Păsăreanu, C.S. (eds.) CAV 2015. LNCS, vol. 9206, pp. 487–495. Springer, Cham (2015). doi:10.1007/978-3-319-21690-4_32

    Chapter  Google Scholar 

  31. Maler, O., Mens, I.-E.: Learning regular languages over large alphabets. In: Ábrahám, E., Havelund, K. (eds.) TACAS 2014. LNCS, vol. 8413, pp. 485–499. Springer, Heidelberg (2014). doi:10.1007/978-3-642-54862-8_41

    Chapter  Google Scholar 

  32. Margaria, T., Steffen, B.: Simplicity as a driver for agile innovation. Computer 43(6), 90–92 (2010)

    Article  Google Scholar 

  33. Mealy, G.H.: A method for synthesizing sequential circuits. Bell Syst. Tech. J. 34(5), 1045–1079 (1955)

    Article  MathSciNet  Google Scholar 

  34. Meinke, K., Sindhu, M.A.: Lbtest: a learning-based testing tool for reactive systems. In: Sixth IEEE International Conference on Software Testing, Verification and Validation, ICST 2013, Luxembourg, 18–22 March 2013, pp. 447–454 (2013)

    Google Scholar 

  35. Merten, M., Steffen, B., Howar, F., Margaria, T.: Next generation learnlib. In: Abdulla, P.A., Leino, K.R.M. (eds.) TACAS 2011. LNCS, vol. 6605, pp. 220–223. Springer, Heidelberg (2011). doi:10.1007/978-3-642-19835-9_18

    Chapter  Google Scholar 

  36. Raffelt, H., Merten, M., Steffen, B., Margaria, T.: Dynamic testing via automata learning. Int. J. Softw. Tools Technol. Trans. (STTT) 11(4), 307–324 (2009)

    Article  Google Scholar 

  37. Raffelt, H., Steffen, B., Berg, T., Margaria, T.: LearnLib: a framework for extrapolating behavioral models. Int. J. Softw. Tools Technol. Trans. (STTT) 11(5), 393–407 (2009)

    Article  Google Scholar 

  38. Shahbaz, M., Groz, R.: Analysis and testing of black-box component-based systems by inferring partial models. Softw. Test. Verif. Reliab. 24(4), 253–288 (2014)

    Article  Google Scholar 

  39. Steffen, B., Howar, F., Merten, M.: Introduction to active automata learning from a practical perspective. In: Bernardo, M., Issarny, V. (eds.) SFM 2011. LNCS, vol. 6659, pp. 256–296. Springer, Heidelberg (2011). doi:10.1007/978-3-642-21455-4_8

    Chapter  Google Scholar 

  40. Timmer, M., Brinksma, E., Stoelinga, M.: Model-based testing. In: Software and Systems Safety - Specification and Verification, pp. 1–32. IOS Press (2011)

    Google Scholar 

  41. Tretmans, J.: Model-based testing and some steps towards test-based modelling. In: Bernardo, M., Issarny, V. (eds.) SFM 2011. LNCS, vol. 6659, pp. 297–326. Springer, Heidelberg (2011). doi:10.1007/978-3-642-21455-4_9

    Chapter  Google Scholar 

  42. Tretmans, J., Brinksma, E.: Torx: automated model-based testing. In: Hartman, A., Dussa-Ziegler, K. (ed.) First European Conference on Model-Driven Software Engineering, pp. 31–43, December 2003

    Google Scholar 

  43. Vaandrager, F.: Model learning. Commun. ACM 60(2), 86–95 (2017)

    Article  Google Scholar 

  44. Windmüller, S., Neubauer, J., Steffen, B., Howar, F., Bauer, O.: Active continuous quality control. In: Proceedings of the 16th International ACM Sigsoft Symposium on Component-based Software Engineering, CBSE 2013, pp. 111–120. ACM New York, NY, USA (2013)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Dortmund University of Technology, Dortmund, Germany

    Alexander Bainczyk, Alexander Schieweck, Bernhard Steffen & Falk Howar

Authors
  1. Alexander Bainczyk
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Alexander Schieweck
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Bernhard Steffen
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Falk Howar
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Alexander Bainczyk or Alexander Schieweck .

Editor information

Editors and Affiliations

  1. RWTH Aachen University, Aachen, Germany

    Joost-Pieter Katoen

  2. University of Twente, Enschede, The Netherlands

    Rom Langerak

  3. University of Twente, Enschede, The Netherlands

    Arend Rensink

Rights and permissions

Reprints and permissions

Copyright information

© 2017 Springer International Publishing AG

About this chapter

Cite this chapter

Bainczyk, A., Schieweck, A., Steffen, B., Howar, F. (2017). Model-Based Testing Without Models: The TodoMVC Case Study. In: Katoen, JP., Langerak, R., Rensink, A. (eds) ModelEd, TestEd, TrustEd. Lecture Notes in Computer Science(), vol 10500. Springer, Cham. https://doi.org/10.1007/978-3-319-68270-9_7

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-68270-9_7

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-68269-3

  • Online ISBN: 978-3-319-68270-9

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation