Abstract
In this paper we present learning-based cross-platform conformance testing (LCCT), an approach specifically designed to validate successful system migration. Key to our approach is the combination of (1) adequate user-level system abstraction, (2) higher-order integration of executable test-blocks, and (3) learning-based automatic model inference and comparison. The impact of LCCT will be illustrated along the migration of Springer’s Online Conference Service (OCS) from a browser-based implementation to using a RESTful web service API. Continuous LCCT allowed us in particular to systematically pinpoint spots where the original OCS depended on browser-based access control mechanisms, to eliminate them, and thus to maintain the OCS access control policy for the RESTFul API.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
- 1.
A KS is a KTS with an empty set of actions, an LTS is a KTS with a trivial interpretation I.
- 2.
This case study of the router with its over 20.000 states was a first indication of the scalability of learning-based validation.
- 3.
- 4.
Cf. http://springer.com.
- 5.
- 6.
In Java a method can have a throws clause to define exceptions that may arise during execution.
- 7.
Methods or functions without a return type are called void in many C-derived languages like, e.g., Java.
- 8.
The notion class may refer to a class, an interface, an enumeration as well as an annotation.
- 9.
The outputs of a start SIB represent the input parameters of the respective process according to the differentiation between actual and formal parameters of functions in programming languages.
- 10.
LearnLib is available at http://www.learnlib.de.
References
Aarts, F., Jonsson, B., Uijen, J.: Generating models of infinite-state communication protocols using regular inference with abstraction. In: Petrenko, A., Simão, A., Maldonado, J.C. (eds.) ICTSS 2010. LNCS, vol. 6435, pp. 188–204. Springer, Heidelberg (2010). doi:10.1007/978-3-642-16573-3_14
Aarts, F., Schmaltz, J., Vaandrager, F.: Inference and abstraction of the biometric passport. In: Margaria, T., Steffen, B. (eds.) ISoLA 2010. LNCS, vol. 6415, pp. 673–686. Springer, Heidelberg (2010). doi:10.1007/978-3-642-16558-0_54
Activiti Team: Activiti BPM Platform (2012). http://www.activiti.org/
Angluin, D.: Learning regular sets from queries and counterexamples. Inf. Comput. 75(2), 87–106 (1987)
Bauer, O., Neubauer, J., Steffen, B., Howar, F.: Reusing system states by active learning algorithms. In: Moschitti, A., Scandariato, R. (eds.) EternalS 2011. CCIS, vol. 255, pp. 61–78. Springer, Heidelberg (2012). doi:10.1007/978-3-642-28033-7_6
Berg, T., Grinchtein, O., Jonsson, B., Leucker, M., Raffelt, H., Steffen, B.: On the correspondence between conformance testing and regular inference. In: Cerioli, M. (ed.) FASE 2005. LNCS, vol. 3442, pp. 175–189. Springer, Heidelberg (2005). doi:10.1007/978-3-540-31984-9_14
Beydeda, S., Gruhn, V.: Integrating white- and black-box techniques for class-level regression testing. In: COMPSAC 2001, pp. 357–362. IEEE Computer Society, Washington, DC (2001)
Bossert, G., Hiet, G., Henin, T.: Modelling to simulate botnet command and control protocols for the evaluation of network intrusion detection systems. In: SAR-SSI 2011, pp. 1–8. IEEE Computer Society (2011)
Chen, Y., Probert, R.L., Ural, H.: Model-based regression test suite generation using dependence analysis. In: A-MOST 2007, pp. 54–62. ACM, New York (2007)
Dadam, P., et al.: From ADEPT to aristaflow BPM suite: a research vision has become reality. In: Rinderle-Ma, S., Sadiq, S., Leymann, F. (eds.) BPM 2009. LNBIP, vol. 43, pp. 529–531. Springer, Heidelberg (2010). doi:10.1007/978-3-642-12186-9_50
Doedt, M., Steffen, B.: An evaluation of service integration approaches of business process management systems. In: 2012 35th IEEE Software Engineering Workshop (SEW) (2012)
Fielding, R.T., Taylor, R.N.: Principled design of the modern web architecture. In: Ghezzi, C., Jazayeri, M., Wolf, A.L., (eds.) ICSE, pp. 407–416. ACM (2000)
Forman, I.R., Forman, N.: Java Reflection in Action (In Action series). Manning Publications Co., Greenwich (2004)
Gnesi, S., Margaria, T.: Formal Methods for Industrial Critical Systems: A Survey of Applications. Wiley, New York (2012)
Hagerer, A., Hungar, H., Margaria, T., Niese, O., Steffen, B., Ide, H.-D.: Demonstration of an operational procedure for the model-based testing of CTI systems. In: Kutsche, R.-D., Weber, H. (eds.) FASE 2002. LNCS, vol. 2306, pp. 336–339. Springer, Heidelberg (2002). doi:10.1007/3-540-45923-5_25
Hagerer, A., Hungar, H., Niese, O., Steffen, B.: Model generation by moderated regular extrapolation. In: Kutsche, R.-D., Weber, H. (eds.) FASE 2002. LNCS, vol. 2306, pp. 80–95. Springer, Heidelberg (2002). doi:10.1007/3-540-45923-5_6
Howar, F., Bauer, O., Merten, M., Steffen, B., Margaria, T.: The teachers’ crowd: The impact of distributed oracles on active automata learning. In: Hähnle, R., Knoop, J., Margaria, T., Schreiner, D., Steffen, B. (eds.) ISoLA 2012, pp. 232–247. Springer, Heidelberg (2012)
Isberner, M., Howar, F., Steffen, B.: The TTT algorithm: a redundancy-free approach to active automata learning. In: Bonakdarpour, B., Smolka, S.A. (eds.) RV 2014. LNCS, vol. 8734, pp. 307–322. Springer, Heidelberg (2014). doi:10.1007/978-3-319-11164-3_26
Isberner, M., Howar, F., Steffen, B.: The open-source learnlib. In: Kroening, D., Păsăreanu, C.S. (eds.) CAV 2015. LNCS, vol. 9206, pp. 487–495. Springer, Heidelberg (2015). doi:10.1007/978-3-319-21690-4_32
Jung, G., Margaria, T., Nagel, R., Schubert, W., Steffen, B., Voigt, H.: SCA and jABC: bringing a service-oriented paradigm to web-service construction. In: Margaria, T., Steffen, B. (eds.) ISoLA 2008. CCIS, vol. 17, pp. 139–154. Springer, Heidelberg (2008). doi:10.1007/978-3-540-88479-8_11
Jörges, S., Lamprecht, A.-L., Margaria, T., Schaefer, I., Steffen, B.: A constraint-based variability modeling framework. STTT 14, 511–530 (2012)
Karusseit, M., Margaria, T.: A web-based runtime-reconfigurable role management service. In: 2006 2nd International Workshop on Automated Specification and Verification of Web Systems, WWV 2006, pp. 53–60. IEEE (2007)
Korel, B., Al-Yami, A.M.: Automated regression test generation. SIGSOFT Softw. Eng. Notes 23(2), 143–152 (1998)
Margaria, T.: Service is in the eyes of the beholder. IEEE Comput. 40, 33–37 (2007)
Margaria, T., Niese, O., Raffelt, H., Steffen, B.: Efficient test-based model generation for legacy reactive systems. In: Ninth IEEE International Proceedings of the High-Level Design Validation and Test Workshop, HLDVT 2004, pp. 95–100. IEEE Computer Society, Washington, DC (2004)
Margaria, T., Steffen, B.: Lightweight coarse-grained coordination: a scalable system-level approach. STTT 5(2–3), 107–123 (2004)
Margaria, T., Steffen, B.: Agile IT: thinking in user-centric models. In: Margaria, T., Steffen, B. (eds.) ISoLA 2008. CCIS, vol. 17, pp. 490–502. Springer, Heidelberg (2008). doi:10.1007/978-3-540-88479-8_35
Margaria, T., Steffen, B.: Business process modeling in the jABC: the one-thing approach. In: Handbook of Research on Business Process Modeling, pp. 1–26. IGI Global (2009)
Margaria, T., Steffen, B.: Continuous model-driven engineering. IEEE Comput. 42(10), 106–109 (2009)
Margaria, T., Steffen, B.: Simplicity as a driver for agile innovation. Computer 43(6), 90–92 (2010)
Margaria, T., Steffen, B.: Service-orientation: conquering complexity with XMDD. In: Hinchey, M., Koyle, L. (eds.) Conquering Complexity, pp. 217–236. Springer, London (2012)
Merten, M., Steffen, B., Howar, F., Margaria, T.: Next generation learnlib. In: Abdulla, P.A., Leino, K.R.M. (eds.) TACAS 2011. LNCS, vol. 6605, pp. 220–223. Springer, Heidelberg (2011). doi:10.1007/978-3-642-19835-9_18
Müller-Olm, M., Schmidt, D., Steffen, B.: Model-checking: a tutorial introduction. In: SAS, pp. 330–354 (1999)
Neubauer, J.: Higher-order process engineering. Ph.D. thesis, Technische Universität Dortmund (2014)
Neubauer, J.: Higher-order process engineering: the technical background. Technical report, Technische Universität Dortmund, April 2014
Neubauer, J., Frohme, M., Steffen, B., Margaria, T.: Prototype-driven development of web applications with DyWA. In: Margaria, T., Steffen, B. (eds.) ISoLA 2014. LNCS, vol. 8802, pp. 56–72. Springer, Heidelberg (2014). doi:10.1007/978-3-662-45234-9_5
Neubauer, J., Steffen, B.: Plug&play higher-order process integration. Computer (2013)
Neubauer, J., Steffen, B.: Second-order servification. In: Herzwurm, G., Margaria, T. (eds.) ICSOB 2013. LNBIP, vol. 150, pp. 13–25. Springer, Heidelberg (2013). doi:10.1007/978-3-642-39336-5_2
Neubauer, J., Steffen, B., Margaria, T.: Higher-order process modeling: product-lining, variability modeling and beyond. arXiv preprint. arXiv:1309.5143 (2013)
Neubauer, J., Windmüller, S., Steffen, B.: Risk-based testing via active continuous quality control. Int. J. STTT 16(5), 569–591 (2014)
OMG. Business Process Model and Notation (BPMN) Version 2.0 (2011). http://www.omg.org/spec/BPMN/2.0/
Oracle: Javadoc API. http://docs.oracle.com/javase/1.5.0/docs/guide/javadoc/doclet/spec/index.html
Raffelt, H., Merten, M., Steffen, B., Margaria, T.: Dynamic testing via automata learning. Int. J. STTT 11(4), 307–324 (2009)
Raffelt, H., Steffen, B., Berg, T.: Learnlib: a library for automata learning and experimentation. In: FMICS 2005, pp. 62–71. ACM (2005)
Raffelt, H., Steffen, B., Berg, T., Margaria, T.: LearnLib: a framework for extrapolating behavioral models. Int. J. STTT 11(5), 393–407 (2009)
RedHat Software - JBoss. jBPM Website (2012). http://www.jboss.org/jbpm
Rivest, R.L., Schapire, R.E.: Inference of finite automata using homing sequences. Inf. Comput. 103(2), 299–347 (1993)
Rothermel, G., Harrold, M.J., Dedhia, J.: Regression test selection for C++ software. Softw. Test. Verif. Reliab. 10, 77–109 (1999)
Scheer, A.-W., Schneider, K.: ARIS – architecture of integrated information systems. In: Bernus, P., Mertins, K., Schmidt, G. (eds.) Handbook on Architectures of Information Systems, pp. 605–623. Springer, Heidelberg (2006). doi:10.1007/3-540-26661-5_25
Selenium. SeleniumHQ Web application testing system, July 2015. http://seleniumhq.org/
Sestoft, P.: Higher-order functions. In: Sestoft, P. (ed.) Programming Language Concepts. Undergraduate Topics in Computer Science, vol. 50, pp. 77–91. Springer, London (2012)
Shahbaz, M., Shashidhar, K.C., Eschbach, R.: Iterative refinement of specification for component based embedded systems. In: ISSTA 2011, pp. 276–286. ACM (2011)
Steffen, B., Howar, F., Merten, M.: Introduction to active automata learning from a practical perspective. In: Bernardo, M., Issarny, V. (eds.) SFM 2011. LNCS, vol. 6659, pp. 256–296. Springer, Heidelberg (2011). doi:10.1007/978-3-642-21455-4_8
Steffen, B., Margaria, T.: METAframe in practice: design of intelligent network services. In: Olderog, E.-R., Steffen, B. (eds.) Correct System Design. LNCS, vol. 1710, pp. 390–415. Springer, Heidelberg (1999). doi:10.1007/3-540-48092-7_17
Steffen, B., Margaria, T., Braun, V., Kalt, N.: Hierarchical service definition. In: Annual Review of Communication, pp. 847–856. International Engineering Consortium, IEC, Chicago (1997)
Steffen, B., Margaria, T., Nagel, R., Jörges, S., Kubczak, C.: Model-driven development with the jABC. In: Bin, E., Ziv, A., Ur, S. (eds.) HVC 2006. LNCS, vol. 4383, pp. 92–108. Springer, Heidelberg (2007). doi:10.1007/978-3-540-70889-6_7
Steffen, B., Neubauer, J.: Simplified validation of emergent systems through automata learning-based testing. In: SEW 2011 (2011)
Tretmans, J.: Model based testing with labelled transition systems. In: Hierons, R.M., Bowen, J.P., Harman, M. (eds.) Formal Methods and Testing. LNCS, vol. 4949, pp. 1–38. Springer, Heidelberg (2008). doi:10.1007/978-3-540-78917-8_1
Tretmans, J.: Model-based testing and some steps towards test-based modelling. In: Bernardo, M., Issarny, V. (eds.) SFM 2011. LNCS, vol. 6659, pp. 297–326. Springer, Heidelberg (2011). doi:10.1007/978-3-642-21455-4_9
Utting, M., Pretschner, A., Legeard, B.: A taxonomy of model-based testing approaches. Softw. Test. Verif. Reliab. 22(5), 297–312 (2012)
W3C. Web Services Description Language (WSDL) Version 2.0 (2007). http://www.w3.org/TR/2007/REC-wsdl20-20070626/
Windmüller, S., Neubauer, J., Steffen, B., Howar, F., Bauer, O.: Active continuous quality control. In: 16th International ACM SIGSOFT Symposium on Component-Based Software Engineering CBSE 2013, pp. 111–120. ACM SIGSOFT, New York (2013)
Wolstencroft, K., et al.: The (my)Grid ontology: bioinformatics service discovery. Int. J. Bioinform. Res. Appl. 3(3), 303–325 (2007)
Xu, L., Dias, M., Richardson, D.: Generating regression tests via model checking. In COMPSAC 2004, pp. 336–341. IEEE Computer Society, Washington, DC (2004)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2016 Springer International Publishing AG
About this paper
Cite this paper
Neubauer, J., Steffen, B. (2016). Learning-Based Cross-Platform Conformance Testing. In: Lamprecht, AL. (eds) Leveraging Applications of Formal Methods, Verification, and Validation . ISoLA 2016. Communications in Computer and Information Science, vol 683. Springer, Cham. https://doi.org/10.1007/978-3-319-51641-7_4
Download citation
DOI: https://doi.org/10.1007/978-3-319-51641-7_4
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-51640-0
Online ISBN: 978-3-319-51641-7
eBook Packages: Computer ScienceComputer Science (R0)