research-article

Validity frame concept as effort-cutting technique within the verification and validation of complex cyber-physical systems

Authors:

Bert Van Acker,

Bentley James Oakes,

Mehrdad Moradi,

Paul Demeulenaere,

Joachim DenilAuthors Info & Claims

MODELS '20: Proceedings of the 23rd ACM/IEEE International Conference on Model Driven Engineering Languages and Systems: Companion Proceedings

Article No.: 80, Pages 1 - 10

https://doi.org/10.1145/3417990.3419226

Published: 26 October 2020 Publication History

Abstract

The increasing performance demands and certification needs of complex cyber-physical systems (CPS) raise the complexity of the engineering process, not only within the development phase, but also in the Verification and Validation (V&V) phase. A proven technique to handle the complexity of CPSs is Model-Based Design (MBD). Nevertheless, the verification and validation of complex CPSs is still an exhaustive process and the usability of the models to front-load V&V activities heavily depends on the knowledge of the models and the correctness of the conducted virtual experiments. In this paper, we explore how the effort (and cost) of the V&V phase of the engineering process of complex CPSs can be reduced by enhancing the knowledge about the system components, and explicitly capturing it within their corresponding validity frame. This effort reduction originates from exploiting the captured system knowledge to generate efficient V&V processes and by automating activities at different model life stages, such as the setup and execution of boundary-value or fault-injection tests. This will be discussed in the context of a complex CPS: a safety-critical adaptive cruise control system.

References

[1]

Jean Arlat, Yves Crouzet, Johan Karlsson, Peter Folkesson, Emmerich Fuchs, and Günther H Leber. 2003. Comparison of physical and software-implemented fault injection techniques. IEEE Trans. Comput. 52, 9 (2003), 1115--1133.

Digital Library

[2]

Barry W Boehm. 1984. Software engineering economics. IEEE transactions on Software Engineering 1 (1984), 4--21.

[3]

Christopher Brooks, Chih-Hong Cheng, Thomas Huining Feng, Edward A Lee, and Reinhard Von Hanxleden. 2008. Model engineering using multimodeling. In Proceedings of the 1st International Workshop on Model Co-Evolution and Consistency Management (MCCM'08).

[4]

Domenico Cotroneo and Roberto Natella. 2013. Fault injection for software certification. IEEE Security & Privacy 11, 4 (2013), 38--45.

Digital Library

[5]

Joachim Denil, Stefan Klikovits, Pieter J Mosterman, Antonio Vallecillo, and Hans Vangheluwe. 2017. The experiment model and validity frame in M&S. In Proceedings of the Symposium on Theory of Modeling & Simulation. 1--12.

Digital Library

[6]

Christian Dietrich, Achim Schmider, Oskar Pusz, Guillermo Payá Vayá, and Daniel Lohmann. 2018. Cross-Layer Fault-Space Pruning for Hardware-Assisted Fault Injection. In 2018 55th ACM/ESDA/IEEE Design Automation Conference (DAC). 1--6.

[7]

Sanford Friedenthal, Regina Griego, and Mark Sampson. 2007. INCOSE model based systems engineering (MBSE) initiative. In INCOSE 2007 symposium, Vol. 11.

[8]

Lex Heerink and Ed Brinksma. 1995. Validation in context. In International Conference on Protocol Specification, Testing and Verification. Springer, 221--236.

[9]

Bingchiang Jeng and Elaine J Weyuker. 1994. A simplified domain-testing strategy. ACM Transactions on Software Engineering and Methodology (TOSEM) 3, 3 (1994), 254--270.

Digital Library

[10]

Edward A. Lee. 2008. Cyber Physical Systems: Design Challenges. In 2008 11th IEEE International Symposium on Object and Component-Oriented Real-Time Distributed Computing (ISORC) (00, Vol. 3), Reginald N. Smythe and Alexander Noble (Eds.). IEEE, 363--369.

Digital Library

[11]

Edward A Lee. 2017. Plato and the Nerd: The Creative Partnership of Humans and Technology. MIT Press.

[12]

David Q Mayne, James B Rawlings, Christopher V Rao, and Pierre OM Scokaert. 2000. Constrained model predictive control: Stability and optimality. Automatica 36, 6 (2000), 789--814.

Digital Library

[13]

Xiankai Meng, Qingping Tan, Zeming Shao, Nan Zhang, Jianjun Xu, and Haoyu Zhang. 2018. Optimization methods for the fault injection tool SEInjector. In 2018 International Conference on Information and Computer Technologies (ICICT). 31--35.

[14]

Mehrdad Moradi, Cláudio Gomes, Bentley James Oakes, and Joachim Denil. 2019. Optimizing Fault Injection in FMI Co-Simulation through Sensitivity Partitioning. In Proceedings of the 2019 Summer Simulation Conference (Berlin, Germany) (SummerSim âĂ&Zacute;19). Society for Computer Simulation International, San Diego, CA, USA, Article 32, 12 pages.

Digital Library

[15]

Mehrdad Moradi, Bentley James Oakes, Mustafa Saraoglu, Andrey Morozov, Klaus Janschek, and Joachim Denil. 2020. Exploring Fault Parameter Space Using Reinforcement Learning-based Fault Injection. In 2020 50th Annual IEEE/IFIP International Conference on Dependable Systems and Networks Workshops (DSN-W). 102--109.

[16]

Mehrdad Moradi, Bert Van Acker, Ken Vanherpen, and Joachim Denil. 2019. Model-Implemented Hybrid Fault Injection for Simulink (Tool Demonstrations). In Cyber Physical Systems. Model-Based Design, Roger Chamberlain, Walid Taha, and Martin Törngren (Eds.). Springer International Publishing, Cham, 71--90.

[17]

Peter Mosterman. 2007. Model-Based Design of Embedded Systems. In 2007 IEEE International Conference on Microelectronic Systems Education (MSE'07). 3--3.

Digital Library

[18]

Glenford J Myers, Corey Sandler, and Tom Badgett. 2011. The art of software testing. John Wiley & Sons.

[19]

Rakesh Rana, Miroslaw Staron, Christian Berger, Jörgen Hansson, Martin Nilsson, and Fredrik Törner. 2013. Increasing Efficiency of ISO 26262 Verification and Validation by Combining Fault Injection and Mutation Testing with Model based Development. ICSOFT 2013 (2013), 251--257.

[20]

Artur Schmidt, Umut Durak, and Thorsten Pawletta. 2016. Model-based testing methodology using system entity structures for MATLAB/Simulink models. SIMULATION 92, 8 (2016), 729--746.

Digital Library

[21]

Mamadou K. Traore and Alexandre Muzy. 2006. Capturing the dual relationship between simulation models and their context. Simulation Modelling Practice and Theory Volume 14, Issue 2 (2006), Pages 126--142.

[22]

FW Vaandrager. 2006. Does it Pay Off? Model-Based Verification and Validation of Embedded Systems! PROGRESS White papers 2006 (2006), 43--66.

[23]

Bert Van Acker, Paul De Meulenaere, Joachim Denil, Yuri Durodie, Alexander Van Bellinghen, and Kris Vanstechelman. 2019. Valid (Re-)Use of Models-of-the-Physics in Cyber-Physical Systems Using Validity Frames. In 2019 Spring Simulation Conference (SpringSim). IEEE, 1--12.

[24]

Bert Van Acker, Joachim Denil, Paul De Meulenaere, Bjorn Aelvoet, Dries Mahieu, and Jan Van Den Oudenhoven. 2018. Generation of test strategies for Model-based Functional Safety testing using an Artifact-centric approach. In Proceedings of MODELS 2018 Workshops Copenhagen, Denmark, October, 14, 2018/Hebig, Regina [edit.]. 563--569.

[25]

Simon Van Mierlo, Bentley James Oakes, Bert Van Acker, Raheleh Eslampanah, Joachim Denil, and Hans Vangheluwe. 2020. Exploring Validity Frames in Practice. http://msdl.cs.mcgill.ca/people/bentley/research/VanMierlo2020.pdf. Proceedings of International Conference on Systems Modelling and Management (ICSMM (2020). Accepted.

[26]

Bernard P Zeigler, Tag Gon Kim, and Herbert Praehofer. 2000. Theory of modeling and simulation. Academic press.

Cited By

Vanommeslaeghe YVan Acker BCornelis MDe Meulenaere P(2023)Towards Continuous Verification and Validation of Multi-Domain System Designs2023 ACM/IEEE International Conference on Model Driven Engineering Languages and Systems Companion (MODELS-C)10.1109/MODELS-C59198.2023.00086(495-499)Online publication date: 1-Oct-2023
https://doi.org/10.1109/MODELS-C59198.2023.00086
Vanommeslaeghe YCeulemans Dvan Acker BDenil JDerammelaere SDe Meulenaere PKühn TSousa V(2022)Validation and uncertainty in model-based design space explorationProceedings of the 25th International Conference on Model Driven Engineering Languages and Systems: Companion Proceedings10.1145/3550356.3561581(702-711)Online publication date: 23-Oct-2022
https://dl.acm.org/doi/10.1145/3550356.3561581
Mertens JDenil J(2021)The Digital Twin as a Common Knowledge Base in DevOps to Support Continuous System EvolutionComputer Safety, Reliability, and Security. SAFECOMP 2021 Workshops10.1007/978-3-030-83906-2_12(158-170)Online publication date: 7-Sep-2021
https://dl.acm.org/doi/10.1007/978-3-030-83906-2_12

Index Terms

Validity frame concept as effort-cutting technique within the verification and validation of complex cyber-physical systems
1. Computer systems organization
  1. Embedded and cyber-physical systems
2. Computing methodologies
  1. Modeling and simulation
    1. Model development and analysis
      1. Model verification and validation

Recommendations

Verification and Validation in Cyber Physical Systems: Research Challenges and a Way Forward
SESCPS '15: Proceedings of the 2015 IEEE/ACM 1st International Workshop on Software Engineering for Smart Cyber-Physical Systems

It is widely held that debugging cyber-physical systems (CPS) is challenging, to date, empirical studies investigating research challenges in CPS verification and validation have not been done. As a result, the exact challenges facing CPS developers in ...
Verification and validation in cyber physical systems: research challenges and a way forward
SEsCPS '15: Proceedings of the First International Workshop on Software Engineering for Smart Cyber-Physical Systems

It is widely held that debugging cyber-physical systems (CPS) is challenging; to date, empirical studies investigating research challenges in CPS verification and validation have not been done. As a result, the exact challenges facing CPS developers in ...
Modeling Automotive Cyber Physical Systems
DCABES '13: Proceedings of the 2013 12th International Symposium on Distributed Computing and Applications to Business, Engineering & Science

Automotive cyber physical systems (CPSs) involve interactions between software controllers, communication networks, and physical devices. These systems are among the most complex cyber physical systems being designed by humans. However, automotive cyber ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

MODELS '20: Proceedings of the 23rd ACM/IEEE International Conference on Model Driven Engineering Languages and Systems: Companion Proceedings

October 2020

713 pages

ISBN:9781450381352

DOI:10.1145/3417990

Conference Chairs:
Esther Guerra
Universidad Autonoma de Madrid, Spain
,
Ludovico Iovino
Gran Sasso Science Institute, Italy

Copyright © 2020 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

SIGSOFT: ACM Special Interest Group on Software Engineering

In-Cooperation

IEEE CS

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 26 October 2020

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Qualifiers

Research-article

Funding Sources

Flanders Innovation & Entrepreneurship (VLAIO)

Conference

MODELS '20

Sponsor:

SIGSOFT

MODELS '20: ACM/IEEE 23rd International Conference on Model Driven Engineering Languages and Systems

October 16 - 23, 2020

Virtual Event, Canada

Acceptance Rates

Overall Acceptance Rate 144 of 506 submissions, 28%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

3
Total Citations
View Citations
144
Total Downloads

Downloads (Last 12 months)21
Downloads (Last 6 weeks)2

Reflects downloads up to 02 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Vanommeslaeghe YVan Acker BCornelis MDe Meulenaere P(2023)Towards Continuous Verification and Validation of Multi-Domain System Designs2023 ACM/IEEE International Conference on Model Driven Engineering Languages and Systems Companion (MODELS-C)10.1109/MODELS-C59198.2023.00086(495-499)Online publication date: 1-Oct-2023
https://doi.org/10.1109/MODELS-C59198.2023.00086
Vanommeslaeghe YCeulemans Dvan Acker BDenil JDerammelaere SDe Meulenaere PKühn TSousa V(2022)Validation and uncertainty in model-based design space explorationProceedings of the 25th International Conference on Model Driven Engineering Languages and Systems: Companion Proceedings10.1145/3550356.3561581(702-711)Online publication date: 23-Oct-2022
https://dl.acm.org/doi/10.1145/3550356.3561581
Mertens JDenil J(2021)The Digital Twin as a Common Knowledge Base in DevOps to Support Continuous System EvolutionComputer Safety, Reliability, and Security. SAFECOMP 2021 Workshops10.1007/978-3-030-83906-2_12(158-170)Online publication date: 7-Sep-2021
https://dl.acm.org/doi/10.1007/978-3-030-83906-2_12

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten