Abstract
Context and background: In Production Systems Engineering (PSE), software and systems testing are success-critical along the production automation life cycle to identify defects early and efficiently. Although test automation concepts enable continuous integration and tests during engineering and maintenance, tool chains are often hardwired, less flexible, and inefficient. Thus, there is a need for more flexible tool chains to support verification and validation of control code variants. Objective: In this book chapter, we (a) describe a flexible Test Automation Framework (TAF) to enable continuous integration and tests and (b) provide an adapted maintenance process to enable efficient verification and validation of control code variants. Method: We build on best practices from Software Engineering and Software Testing to establish a flexible TAF based on Behavior-Driven Testing. We use the Abstract Syntax Tree (AST) as foundation for human-based verification and validation. We developed an initial prototype derived from industry partners and used an Industry 4.0 Testbed for evaluation. Results and conclusion: First results of the prototype implementation with selected testing tools showed the capability of the TAF concept for supporting flexible configurations of testing tool chains. The AST concept can support the human-based verification and validation of control code variants.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
Aurum, A., Petersson, H., & Wohlin, C. (2002). State-of-the-art: Software inspections after 25 years. Journal of Software Testing, Verification and Reliability, 12(3), 133–154.
Beck, K. (2003). Test-driven development: By example. Boston, MA: Addison-Wesley.
Biffl, S., Lüder, A., & Winkler, D. (2016). Multi-disciplinary engineering for Industrie 4.0: Semantic challenges, needs, and capabilities. In S. Biffl & M. Sabou (Eds.), Semantic web for intelligent engineering applications. New York: Springer.
Brambilla, M., Cabot, J., & Wimmer, M. (2012). Model-driven software engineering in practice. San Rafael, CA: Morgan & Claypool.
Broy, M. (2006). The ‘grand challenge’ in informatics: Engineering software-intensive systems. IEEE Computer, 39(10), 72–80.
Duvall, P. M., Matyas, S., & Glover, A. (2007). Continuous integration: Improving software quality and reducing risk. London: Pearson.
Erdogmus, H., Morisio, M., & Torchiano, M. (2005). On the effectiveness of the test-first approach to programming. IEEE Transactions on Software Engineering, 31(3), 226–237.
Gilb, T., Graham, D., & Finzi, S. (1993). Software inspection. Boston, MA: Addison Wesley.
Häser, F., Felderer, M., & Breu, R. (2016). Is business domain language support beneficial for creating test case specifications: A controlled experiment. Information and Software Technology, 79, 52–62.
ISO 29119-2. (2013). Software and systems engineering – Software testing – Part 2: Test processes, ISO/IEC/IEEE291192.
ISO 29119-5. (2016). Software and systems engineering – Software testing – Part 5: Keyword-driven testing, ISO/IEC/IEEE 29119-5.
Jones, J. (2003). Abstract syntax tree implementation idioms. In Proceedings of the 10th Conference on Pattern Languages of Programs (PLOP) (pp. 1–10). North Carolina: The Hillside Group.
Laitenberger, O., & DeBaud, J. M. (2000). An encompassing life cycle centric survey of software inspection. Journal of Systems and Software, 50(1), 5–31.
Laukkanen, E., Itkonen, J., & Lassenius, C. (2017). Problems, causes and solutions when adopting continuous delivery—A systematic literature review. Information and Software Technology, 82, 55–79.
Lindstrom, L., & Jeffries, R. (2003). Extreme programming and agile software development methodologies. In IS management handbook (8th ed.). Abingdon: Taylor & Francis.
Mårtensson, T., Hammarstrom, P., & Bosch, J. (2017). Continuous integration is not about build systems. In Proceedings of the 43rd Euromicro Conference on Software Engineering and Advanced Applications (SEAA). IEEE.
Mårtensson, T., Ståhl, D., & Bosch, J. (2018). Enable more frequent integration of software in industry projects. Journal of Systems and Software, 142, 223–236.
Micallef, M., & Colombo, C. (2015). Lessons learnt from using DSLs for automated software testing. In Proceedings of 8th International Conference on Software Testing, Verification and Validation Workshops (ICSTW). IEEE.
Moser, T., Mordinyi, R., & Winkler, D. (2012). Extending mechatronic objects for automation systems engineering in heterogeneous engineering environments. In Proceedings of the 17th International Conference on Emerging Technologies & Factory Automation (ETFA). IEEE.
Musil, J., Musil, A., Weyns, D., & Biffl, S. (2015). An architecture framework for collective intelligence systems. In Proceedings of the 12th Working IEEE/IFIP Conference on Software Architecture. ACM.
Roche, J. (2013). Adopting DevOps practices in quality assurance. Communications of the ACM, 56(11), 38–43.
Rosen, R., von Wichert, G., Lo, G., & Bettenhausen, K. D. (2015). About the importance of autonomy and digital twins for the future of manufacturing. IFAC-PapersOnLine, 48(3), 567–572.
Sabou, M., Winkler, D., Penzerstadler, P., & Biffl, S. (2018). Verifying conceptual domain models with human computation: A casestudy in software engineering. In Proceedings of the 6th AAAI conference on human computation and crowdsourcing (HCOMP). Zurich: AAAI.
Schatten, A., Biffl, S., Demolsky, M., Gostischa-Fanta, E., Östereicher, T., & Winkler, D. (2010). Best Practice Software-Engineering: Eine praxiserprobte Zusammenstellung von komponentenorientierten Konzepten, Methoden und Werkzeugen. Springer.
Schwaber, K., & Beedle, M. (2002). Agile software development with Scrum. London: Pearson.
Shahin, M., Babar, M. A., & Zhu, L. (2017). Continuous integration, delivery and deployment: A systematic review on approaches, tools, challenges and practices. IEEE Access, 5, 3909–3943.
Soeken, M., Wille, R., & Drechsler, R. (2012). Assisted behavior driven development using natural language processing. In Objects, models, components, patterns, tools (pp. 269–287). Berlin: Springer.
Solis, C., & Wang, X. (2011). A study of the characteristics of behaviour driven development. In Proceedings of the 37th EUROMICRO Conference on Software Engineering and Advanced Applications (SEAA). IEEE.
Spillner, A., Linz, T., & Schaefer, H. (2014). Software testing foundations: A study guide for the certified tester exam. San Rafael, CA: Rocky Nook.
Ståhl, D., & Bosch, J. (2014). Modeling continuous integration practice differences in industry software development. Journal of Systems and Software, 87, 48–59.
Sunindyo, W. D., Moser, T., Winkler, D., & Biffl, S. (2010). Foundations for event-based process analysis in heterogeneous software engineering environments. In Proceedings of the 36th EUROMICRO conference on software engineering and advanced applications (SEAA). IEEE.
Van Bennekum, A., Cockburn, A., Cunningham, W., Fowler, M., Grenning, J., Highsmith, J., Hunt, A., Jeffries, R., Beck, K., & Beedle, M. (2001). Manifesto for agile software development. Alicante: Universidad de Alicante.
Vogel-Heuser, B., Kegel, G., Bender, K., & Wucherer, K. (2009). Global information architecture for industrial automation. In Automatisierungstechnische Praxis (atp). Munich: Oldenbourg-Verlag.
Vogel-Heuser, B., Fay, A., Schaefer, I., & Tichy, M. (2015). Evolution of software in automated production systems: Challenges and research directions. Journal of Systems and Software, 110, 54–84.
Whitehead, J. (2007). Collaboration in software engineering: A roadmap. In Future of software engineering (FOSE). IEEE.
Winkler, D., Wimmer, M., Berardinelli, L., & Biffl, S. (2017a). Towards model quality assurance for multi-disciplinary engineering.Needs, challenges, and solution concepts in an AutomationMLcontext. In S. Biffl, A. Lüder, & D. Gerhard (Eds.), Multi-disciplinary engineering for cyber-physical production systems, chapter 16 (pp. 433–457). Cham: Springer.
Winkler, D., Sabou, M., Petrovic, S., Biffl, S., Kalinowski, M., & Carneiro, G. (2017b). Improving model inspection with crowdsourcing. In Proceedings of the 4th International Workshop on Crowdsourcing in Software Engineering. Buenos Aires: ACM/IEEE International Conference on Software Engineering (ICSE).
Winkler, D., Sabou, M., & Biffl, S. (2017c). Improving quality assurance in multidisciplinary engineering environments with semantic technologies. In Quality control and assurance - an ancient Greek term re-mastered. London: Intec Publishing.
Winkler, D., Meixner, K., & Biffl, S. (2018a). Towards flexible and automated testing in production systems engineering projects. In Proceedings of the 23rd International Conference on Emerging Technologies and Factory Automation (ETFA). IEEE.
Winkler, D., Sabou, M., Petrovic, S., Biffl, S., & Kalinowski, M. (2018b). Investigating a distributed and scalable model review process. Centro Latinoamericano de Estudios en Informatica (CLEI), 21(1).
Zhu, Y.-M. (2016). Software reading techniques: Twenty techniques for more effective software review and inspection. New York: Apress.
Acknowledgments
The financial support by the Christian Doppler Research Association, the Austrian Federal Ministry for Digital and Economic Affairs, and the National Foundation for Research, Technology and Development is gratefully acknowledged. The research done by Petr Novak has been supported by the DAMiAS project funded by the Technology Agency of the Czech Republic.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this chapter
Cite this chapter
Winkler, D., Meixner, K., Novak, P. (2019). Efficient and Flexible Test Automation in Production Systems Engineering. In: Biffl, S., Eckhart, M., Lüder, A., Weippl, E. (eds) Security and Quality in Cyber-Physical Systems Engineering. Springer, Cham. https://doi.org/10.1007/978-3-030-25312-7_9
Download citation
DOI: https://doi.org/10.1007/978-3-030-25312-7_9
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-25311-0
Online ISBN: 978-3-030-25312-7
eBook Packages: Computer ScienceComputer Science (R0)