Skip to main content
SpringerLink
Log in

Learning-Based Cross-Platform Conformance Testing

  • Conference paper
  • First Online:
Leveraging Applications of Formal Methods, Verification, and Validation (ISoLA 2016)

Part of the book series: Communications in Computer and Information Science ((CCIS,volume 683))

Included in the following conference series:

  • 332 Accesses

Abstract

In this paper we present learning-based cross-platform conformance testing (LCCT), an approach specifically designed to validate successful system migration. Key to our approach is the combination of (1) adequate user-level system abstraction, (2) higher-order integration of executable test-blocks, and (3) learning-based automatic model inference and comparison. The impact of LCCT will be illustrated along the migration of Springer’s Online Conference Service (OCS) from a browser-based implementation to using a RESTful web service API. Continuous LCCT allowed us in particular to systematically pinpoint spots where the original OCS depended on browser-based access control mechanisms, to eliminate them, and thus to maintain the OCS access control policy for the RESTFul API.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    A KS is a KTS with an empty set of actions, an LTS is a KTS with a trivial interpretation I.

  2. 2.

    This case study of the router with its over 20.000 states was a first indication of the scalability of learning-based validation.

  3. 3.

    http://hope.scce.info.

  4. 4.

    Cf. http://springer.com.

  5. 5.

    http://maven.apache.org.

  6. 6.

    In Java a method can have a throws clause to define exceptions that may arise during execution.

  7. 7.

    Methods or functions without a return type are called void in many C-derived languages like, e.g., Java.

  8. 8.

    The notion class may refer to a class, an interface, an enumeration as well as an annotation.

  9. 9.

    The outputs of a start SIB represent the input parameters of the respective process according to the differentiation between actual and formal parameters of functions in programming languages.

  10. 10.

    LearnLib is available at http://www.learnlib.de.

References

  1. Aarts, F., Jonsson, B., Uijen, J.: Generating models of infinite-state communication protocols using regular inference with abstraction. In: Petrenko, A., Simão, A., Maldonado, J.C. (eds.) ICTSS 2010. LNCS, vol. 6435, pp. 188–204. Springer, Heidelberg (2010). doi:10.1007/978-3-642-16573-3_14

    Chapter  Google Scholar 

  2. Aarts, F., Schmaltz, J., Vaandrager, F.: Inference and abstraction of the biometric passport. In: Margaria, T., Steffen, B. (eds.) ISoLA 2010. LNCS, vol. 6415, pp. 673–686. Springer, Heidelberg (2010). doi:10.1007/978-3-642-16558-0_54

    Chapter  Google Scholar 

  3. Activiti Team: Activiti BPM Platform (2012). http://www.activiti.org/

  4. Angluin, D.: Learning regular sets from queries and counterexamples. Inf. Comput. 75(2), 87–106 (1987)

    Article  MathSciNet  MATH  Google Scholar 

  5. Bauer, O., Neubauer, J., Steffen, B., Howar, F.: Reusing system states by active learning algorithms. In: Moschitti, A., Scandariato, R. (eds.) EternalS 2011. CCIS, vol. 255, pp. 61–78. Springer, Heidelberg (2012). doi:10.1007/978-3-642-28033-7_6

    Chapter  Google Scholar 

  6. Berg, T., Grinchtein, O., Jonsson, B., Leucker, M., Raffelt, H., Steffen, B.: On the correspondence between conformance testing and regular inference. In: Cerioli, M. (ed.) FASE 2005. LNCS, vol. 3442, pp. 175–189. Springer, Heidelberg (2005). doi:10.1007/978-3-540-31984-9_14

    Chapter  Google Scholar 

  7. Beydeda, S., Gruhn, V.: Integrating white- and black-box techniques for class-level regression testing. In: COMPSAC 2001, pp. 357–362. IEEE Computer Society, Washington, DC (2001)

    Google Scholar 

  8. Bossert, G., Hiet, G., Henin, T.: Modelling to simulate botnet command and control protocols for the evaluation of network intrusion detection systems. In: SAR-SSI 2011, pp. 1–8. IEEE Computer Society (2011)

    Google Scholar 

  9. Chen, Y., Probert, R.L., Ural, H.: Model-based regression test suite generation using dependence analysis. In: A-MOST 2007, pp. 54–62. ACM, New York (2007)

    Google Scholar 

  10. Dadam, P., et al.: From ADEPT to aristaflow BPM suite: a research vision has become reality. In: Rinderle-Ma, S., Sadiq, S., Leymann, F. (eds.) BPM 2009. LNBIP, vol. 43, pp. 529–531. Springer, Heidelberg (2010). doi:10.1007/978-3-642-12186-9_50

    Chapter  Google Scholar 

  11. Doedt, M., Steffen, B.: An evaluation of service integration approaches of business process management systems. In: 2012 35th IEEE Software Engineering Workshop (SEW) (2012)

    Google Scholar 

  12. Fielding, R.T., Taylor, R.N.: Principled design of the modern web architecture. In: Ghezzi, C., Jazayeri, M., Wolf, A.L., (eds.) ICSE, pp. 407–416. ACM (2000)

    Google Scholar 

  13. Forman, I.R., Forman, N.: Java Reflection in Action (In Action series). Manning Publications Co., Greenwich (2004)

    MATH  Google Scholar 

  14. Gnesi, S., Margaria, T.: Formal Methods for Industrial Critical Systems: A Survey of Applications. Wiley, New York (2012)

    Book  Google Scholar 

  15. Hagerer, A., Hungar, H., Margaria, T., Niese, O., Steffen, B., Ide, H.-D.: Demonstration of an operational procedure for the model-based testing of CTI systems. In: Kutsche, R.-D., Weber, H. (eds.) FASE 2002. LNCS, vol. 2306, pp. 336–339. Springer, Heidelberg (2002). doi:10.1007/3-540-45923-5_25

    Chapter  Google Scholar 

  16. Hagerer, A., Hungar, H., Niese, O., Steffen, B.: Model generation by moderated regular extrapolation. In: Kutsche, R.-D., Weber, H. (eds.) FASE 2002. LNCS, vol. 2306, pp. 80–95. Springer, Heidelberg (2002). doi:10.1007/3-540-45923-5_6

    Chapter  Google Scholar 

  17. Howar, F., Bauer, O., Merten, M., Steffen, B., Margaria, T.: The teachers’ crowd: The impact of distributed oracles on active automata learning. In: Hähnle, R., Knoop, J., Margaria, T., Schreiner, D., Steffen, B. (eds.) ISoLA 2012, pp. 232–247. Springer, Heidelberg (2012)

    Google Scholar 

  18. Isberner, M., Howar, F., Steffen, B.: The TTT algorithm: a redundancy-free approach to active automata learning. In: Bonakdarpour, B., Smolka, S.A. (eds.) RV 2014. LNCS, vol. 8734, pp. 307–322. Springer, Heidelberg (2014). doi:10.1007/978-3-319-11164-3_26

    Google Scholar 

  19. Isberner, M., Howar, F., Steffen, B.: The open-source learnlib. In: Kroening, D., Păsăreanu, C.S. (eds.) CAV 2015. LNCS, vol. 9206, pp. 487–495. Springer, Heidelberg (2015). doi:10.1007/978-3-319-21690-4_32

    Chapter  Google Scholar 

  20. Jung, G., Margaria, T., Nagel, R., Schubert, W., Steffen, B., Voigt, H.: SCA and jABC: bringing a service-oriented paradigm to web-service construction. In: Margaria, T., Steffen, B. (eds.) ISoLA 2008. CCIS, vol. 17, pp. 139–154. Springer, Heidelberg (2008). doi:10.1007/978-3-540-88479-8_11

    Chapter  Google Scholar 

  21. Jörges, S., Lamprecht, A.-L., Margaria, T., Schaefer, I., Steffen, B.: A constraint-based variability modeling framework. STTT 14, 511–530 (2012)

    Article  Google Scholar 

  22. Karusseit, M., Margaria, T.: A web-based runtime-reconfigurable role management service. In: 2006 2nd International Workshop on Automated Specification and Verification of Web Systems, WWV 2006, pp. 53–60. IEEE (2007)

    Google Scholar 

  23. Korel, B., Al-Yami, A.M.: Automated regression test generation. SIGSOFT Softw. Eng. Notes 23(2), 143–152 (1998)

    Article  Google Scholar 

  24. Margaria, T.: Service is in the eyes of the beholder. IEEE Comput. 40, 33–37 (2007)

    Article  Google Scholar 

  25. Margaria, T., Niese, O., Raffelt, H., Steffen, B.: Efficient test-based model generation for legacy reactive systems. In: Ninth IEEE International Proceedings of the High-Level Design Validation and Test Workshop, HLDVT 2004, pp. 95–100. IEEE Computer Society, Washington, DC (2004)

    Google Scholar 

  26. Margaria, T., Steffen, B.: Lightweight coarse-grained coordination: a scalable system-level approach. STTT 5(2–3), 107–123 (2004)

    Article  Google Scholar 

  27. Margaria, T., Steffen, B.: Agile IT: thinking in user-centric models. In: Margaria, T., Steffen, B. (eds.) ISoLA 2008. CCIS, vol. 17, pp. 490–502. Springer, Heidelberg (2008). doi:10.1007/978-3-540-88479-8_35

    Chapter  Google Scholar 

  28. Margaria, T., Steffen, B.: Business process modeling in the jABC: the one-thing approach. In: Handbook of Research on Business Process Modeling, pp. 1–26. IGI Global (2009)

    Google Scholar 

  29. Margaria, T., Steffen, B.: Continuous model-driven engineering. IEEE Comput. 42(10), 106–109 (2009)

    Article  Google Scholar 

  30. Margaria, T., Steffen, B.: Simplicity as a driver for agile innovation. Computer 43(6), 90–92 (2010)

    Article  Google Scholar 

  31. Margaria, T., Steffen, B.: Service-orientation: conquering complexity with XMDD. In: Hinchey, M., Koyle, L. (eds.) Conquering Complexity, pp. 217–236. Springer, London (2012)

    Chapter  Google Scholar 

  32. Merten, M., Steffen, B., Howar, F., Margaria, T.: Next generation learnlib. In: Abdulla, P.A., Leino, K.R.M. (eds.) TACAS 2011. LNCS, vol. 6605, pp. 220–223. Springer, Heidelberg (2011). doi:10.1007/978-3-642-19835-9_18

    Chapter  Google Scholar 

  33. Müller-Olm, M., Schmidt, D., Steffen, B.: Model-checking: a tutorial introduction. In: SAS, pp. 330–354 (1999)

    Google Scholar 

  34. Neubauer, J.: Higher-order process engineering. Ph.D. thesis, Technische Universität Dortmund (2014)

    Google Scholar 

  35. Neubauer, J.: Higher-order process engineering: the technical background. Technical report, Technische Universität Dortmund, April 2014

    Google Scholar 

  36. Neubauer, J., Frohme, M., Steffen, B., Margaria, T.: Prototype-driven development of web applications with DyWA. In: Margaria, T., Steffen, B. (eds.) ISoLA 2014. LNCS, vol. 8802, pp. 56–72. Springer, Heidelberg (2014). doi:10.1007/978-3-662-45234-9_5

    Google Scholar 

  37. Neubauer, J., Steffen, B.: Plug&play higher-order process integration. Computer (2013)

    Google Scholar 

  38. Neubauer, J., Steffen, B.: Second-order servification. In: Herzwurm, G., Margaria, T. (eds.) ICSOB 2013. LNBIP, vol. 150, pp. 13–25. Springer, Heidelberg (2013). doi:10.1007/978-3-642-39336-5_2

    Chapter  Google Scholar 

  39. Neubauer, J., Steffen, B., Margaria, T.: Higher-order process modeling: product-lining, variability modeling and beyond. arXiv preprint. arXiv:1309.5143 (2013)

  40. Neubauer, J., Windmüller, S., Steffen, B.: Risk-based testing via active continuous quality control. Int. J. STTT 16(5), 569–591 (2014)

    Article  Google Scholar 

  41. OMG. Business Process Model and Notation (BPMN) Version 2.0 (2011). http://www.omg.org/spec/BPMN/2.0/

  42. Oracle: Javadoc API. http://docs.oracle.com/javase/1.5.0/docs/guide/javadoc/doclet/spec/index.html

  43. Raffelt, H., Merten, M., Steffen, B., Margaria, T.: Dynamic testing via automata learning. Int. J. STTT 11(4), 307–324 (2009)

    Article  Google Scholar 

  44. Raffelt, H., Steffen, B., Berg, T.: Learnlib: a library for automata learning and experimentation. In: FMICS 2005, pp. 62–71. ACM (2005)

    Google Scholar 

  45. Raffelt, H., Steffen, B., Berg, T., Margaria, T.: LearnLib: a framework for extrapolating behavioral models. Int. J. STTT 11(5), 393–407 (2009)

    Article  Google Scholar 

  46. RedHat Software - JBoss. jBPM Website (2012). http://www.jboss.org/jbpm

  47. Rivest, R.L., Schapire, R.E.: Inference of finite automata using homing sequences. Inf. Comput. 103(2), 299–347 (1993)

    Article  MathSciNet  MATH  Google Scholar 

  48. Rothermel, G., Harrold, M.J., Dedhia, J.: Regression test selection for C++ software. Softw. Test. Verif. Reliab. 10, 77–109 (1999)

    Article  Google Scholar 

  49. Scheer, A.-W., Schneider, K.: ARIS – architecture of integrated information systems. In: Bernus, P., Mertins, K., Schmidt, G. (eds.) Handbook on Architectures of Information Systems, pp. 605–623. Springer, Heidelberg (2006). doi:10.1007/3-540-26661-5_25

    Google Scholar 

  50. Selenium. SeleniumHQ Web application testing system, July 2015. http://seleniumhq.org/

  51. Sestoft, P.: Higher-order functions. In: Sestoft, P. (ed.) Programming Language Concepts. Undergraduate Topics in Computer Science, vol. 50, pp. 77–91. Springer, London (2012)

    Chapter  Google Scholar 

  52. Shahbaz, M., Shashidhar, K.C., Eschbach, R.: Iterative refinement of specification for component based embedded systems. In: ISSTA 2011, pp. 276–286. ACM (2011)

    Google Scholar 

  53. Steffen, B., Howar, F., Merten, M.: Introduction to active automata learning from a practical perspective. In: Bernardo, M., Issarny, V. (eds.) SFM 2011. LNCS, vol. 6659, pp. 256–296. Springer, Heidelberg (2011). doi:10.1007/978-3-642-21455-4_8

    Chapter  Google Scholar 

  54. Steffen, B., Margaria, T.: METAframe in practice: design of intelligent network services. In: Olderog, E.-R., Steffen, B. (eds.) Correct System Design. LNCS, vol. 1710, pp. 390–415. Springer, Heidelberg (1999). doi:10.1007/3-540-48092-7_17

    Chapter  Google Scholar 

  55. Steffen, B., Margaria, T., Braun, V., Kalt, N.: Hierarchical service definition. In: Annual Review of Communication, pp. 847–856. International Engineering Consortium, IEC, Chicago (1997)

    Google Scholar 

  56. Steffen, B., Margaria, T., Nagel, R., Jörges, S., Kubczak, C.: Model-driven development with the jABC. In: Bin, E., Ziv, A., Ur, S. (eds.) HVC 2006. LNCS, vol. 4383, pp. 92–108. Springer, Heidelberg (2007). doi:10.1007/978-3-540-70889-6_7

    Chapter  Google Scholar 

  57. Steffen, B., Neubauer, J.: Simplified validation of emergent systems through automata learning-based testing. In: SEW 2011 (2011)

    Google Scholar 

  58. Tretmans, J.: Model based testing with labelled transition systems. In: Hierons, R.M., Bowen, J.P., Harman, M. (eds.) Formal Methods and Testing. LNCS, vol. 4949, pp. 1–38. Springer, Heidelberg (2008). doi:10.1007/978-3-540-78917-8_1

    Chapter  Google Scholar 

  59. Tretmans, J.: Model-based testing and some steps towards test-based modelling. In: Bernardo, M., Issarny, V. (eds.) SFM 2011. LNCS, vol. 6659, pp. 297–326. Springer, Heidelberg (2011). doi:10.1007/978-3-642-21455-4_9

    Chapter  Google Scholar 

  60. Utting, M., Pretschner, A., Legeard, B.: A taxonomy of model-based testing approaches. Softw. Test. Verif. Reliab. 22(5), 297–312 (2012)

    Article  Google Scholar 

  61. W3C. Web Services Description Language (WSDL) Version 2.0 (2007). http://www.w3.org/TR/2007/REC-wsdl20-20070626/

  62. Windmüller, S., Neubauer, J., Steffen, B., Howar, F., Bauer, O.: Active continuous quality control. In: 16th International ACM SIGSOFT Symposium on Component-Based Software Engineering CBSE 2013, pp. 111–120. ACM SIGSOFT, New York (2013)

    Google Scholar 

  63. Wolstencroft, K., et al.: The (my)Grid ontology: bioinformatics service discovery. Int. J. Bioinform. Res. Appl. 3(3), 303–325 (2007)

    Article  Google Scholar 

  64. Xu, L., Dias, M., Richardson, D.: Generating regression tests via model checking. In COMPSAC 2004, pp. 336–341. IEEE Computer Society, Washington, DC (2004)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Technische Universität Dortmund, Dortmund, Germany

    Johannes Neubauer & Bernhard Steffen

Authors
  1. Johannes Neubauer
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Bernhard Steffen
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Johannes Neubauer .

Editor information

Editors and Affiliations

  1. Lero - Irish Software Research Center, University of Limerick, Limerick, Ireland

    Anna-Lena Lamprecht

Rights and permissions

Reprints and permissions

Copyright information

© 2016 Springer International Publishing AG

About this paper

Cite this paper

Neubauer, J., Steffen, B. (2016). Learning-Based Cross-Platform Conformance Testing. In: Lamprecht, AL. (eds) Leveraging Applications of Formal Methods, Verification, and Validation . ISoLA 2016. Communications in Computer and Information Science, vol 683. Springer, Cham. https://doi.org/10.1007/978-3-319-51641-7_4

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-51641-7_4

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-51640-0

  • Online ISBN: 978-3-319-51641-7

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation