Supporting semi-automatic co-evolution of architecture and fault tree models
Introduction
Quality of service (QoS) attributes such as safety, reliability and performance are crucial for software-intensive systems, e.g., in safety-critical or automation systems. Such systems (e.g. aircraft and automation systems, robotics) are not tolerable with an erroneous design since they are playing fundamental roles in human lives. Therefore quality evaluation during the development of systems from design time to runtime is inevitable. A rigorous quality evaluation is among the key methods for the dependable engineering of such systems. To that end, model-based approaches have been proposed which construct quality evaluation models from system models to gain knowledge about the quality of a system by checking these models against formally specified quality requirements (Grunske and Han, 2008).
In model-based quality evaluation, the consistency of the involved models is of utmost importance. For example, the failures of an architectural component must be adequately considered in an associated fault tree model. While this consistency requirement can be reasonably met for a particular snapshot of a system, quality evaluation models typically become outdated when the system evolves, i.e., quality evaluation models and system models evolve in an inconsistent way. As a consequence, quality evaluation leads to unexpected and highly improper results. An example in the context of hazard analysis of component-based embedded systems is the addition of a new port for a sensor of a component without a corresponding addition of the sensor failures in the relevant fault trees. This will clearly lead to wrong hazard analysis results. Hence, loosely inter-related models such as architectural models and quality evaluation models should consistently evolve in parallel, a phenomenon to which we refer to as (consistent) model co-evolution in the remainder of this paper.
Since loosely inter-related models are typically changed in isolation of each other, one adequate approach to support developers is model synchronization, i.e., the task of adapting a model in response to changes in one of its inter-related counterparts in order to achieve consistent co-evolution. In general, however, achieving this kind of model co-evolution is not a straightforward task (Mens et al., 2005): quality evaluation models cannot be fully generated from system models, and most relations between the elements of the different models are not simple one-to-one correspondences. In other words, achieving consistent co-evolution cannot be fully automated as usually assumed by existing approaches to model synchronization (see Section 2 for a more detailed discussion of related work in this area Getir et al., 2013). At best, developers may be supported by recommending possible synchronization actions, e.g. as in the model-based (co-)evolution framework known as CoWolf (Getir et al., 2015). To achieve consistent co-evolution, CoWolf follows a rule-based approach where incremental model transformations are used to recommend both intra- and inter-model change actions. However, since the adequacy of the recommendations strongly depends on the transformation rules being used by the tool, the evolution problem is shifted to the engineering of proper transformation rules. These should capture evolution and co-evolution steps being considered as useful by the developers using the tool.
In this paper, we tackle this problem of engineering proper transformation rule sets for an important class of models in the context of model-based hazard analysis of software-intensive systems, namely system architecture models and fault tree models. We extend our previous work Getir et al. (2013) on the evolution of the so-called Pick and Place Unit (PPU) (Legat et al., 2013), a case study from the automation engineering domain which is commonly used in the German priority program “Design for Future – Managed Software Evolution” (Goltz et al., 2015). To study co-evolution in terms of the PPU, we created consistent software architecture and fault tree models for all safety-relevant evolution scenarios. The contributions of this paper are:
- 1.
A thorough quantitative analysis of the evolution scenarios with respect to the co-evolution of the models, i.e., how changes in one model affect changes in the other model. We show that the models do not co-evolve in a systematic, automatable way and instead expertise of the developer is required to achieve co-evolution. This is a minor contribution which, while not being exceedingly surprising, confirms the findings of previous research in this context.
- 2.
The major contribution is a set of model transformation rules for (1) the independent evolution of software architecture and fault tree models and (2) synchronization of one model based on changes in another model ensuring a correct co-evolution of both models. In the evaluation of the rules, we show that the presented set of model transformations is complete, i.e., it supports performing all co-evolutions of the case study scenarios, and improves the task efficiency(cf. Quality in Use, ISO/IEC, 2001) by reducing the amount of required model transformation applications to realize the co-evolution by, on average, 52% compared to visual editing operations and 85% compared to atomic model changes. Additionally, we implemented these rules in the tool CoWolf (Getir et al., 2015) to enable the co-evolution of fault trees and software architecture models.
To enable reproducibility of our results, we make all models for the scenarios, the set of model transformation rules as well as the code for the evaluation publicly available.1
The remainder of this paper is structured as follows. In the next section, we discuss related work in the areas of safety evaluation models and their automatic generation as well as different approaches to achieve consistent co-evolution of inter-related models. Section 3 briefly sketches the used modeling and model transformation languages. Thereafter, we present the results of the quantitative analysis of the co-evolution of the models of the case study in Section 4. Section 5 contains a description of the developed inter- and intra-model transformation rules for architecture and fault tree models. We evaluate this set of model transformation rules in Section 6. Finally, we conclude and present an outlook on planned future work in incremental analysis of quality models and supporting the developer selecting model transformations by analyzing historic developer decisions on co-evolutions. The appendix contains a thorough description of all evolution scenarios and their impact on system architecture and fault tree models.
Section snippets
Related work
The work presented in this paper draws from two separate research areas, namely the work on automatic/semi-automatic generation of safety evaluation models in the architectural design phase and general approaches that aim at keeping dependent models consistent while individual models evolve.
Safety evaluation models and their automatic generation: Several evaluation models have been proposed to facilitate a quantitative safety analysis based on architectural specifications (Grunske and Han, 2008
Background
Our work is based on model-driven software engineering, particularly, modeling software architectures and fault trees as well as specifying model transformations. Therefore, we briefly introduce our two modeling languages including their meta-models as well as the Henshin transformation language (Arendt et al., 2010) which we use to specify model transformations.
Case study on architecture and fault tree co-evolution
The system we studied is an industrial production plant, called Pick and Place Unit (PPU). The PPU is a factory automation system that mimics an industrial robot that moves work pieces (WPs) between different working positions where they are stored or processed. Such systems are an interesting case for evolution since they contain mechanical parts, electrical parts, and software parts. All these parts can be evolved individually or in combination. Additionally, these systems are also typically
Supporting co-evolution
In Section 4, we presented the analysis results and conclude that change actions between system architecture and fault trees are not straightforward. For this reason, developers can only be assisted by a recommender system that offers a set of meaningful transformations, which is rich enough to consistently co-evolve system architecture and fault tree models. Specifications of such transformations can be integrated into a rule-based framework such as the CoWolf tool (Getir et al., 2015)
Evaluation
The quality of a co-evolution framework such as CoWolf strongly depends on the quality of the transformation rules being offered to developers as interactive editing commands. Thus, we investigate two major quality aspects of our set of manually defined transformation rules which must be (i) complete in the sense that every couple (SAi, FTi) can be evolved to (SAip, FTip) in a consistent way, and (ii) helpful in the sense that this evolution can be achieved by a developer with minimal effort,
Conclusion
In this paper, we have thoroughly analyzed the co-evolution of architecture models and fault trees for a factory automation system called Pick and Place Unit as an extension of our previous work (Getir et al., 2013).
As a major contribution, we provided a set of model transformation rules for achieving co-evolution of software architecture and fault tree models ensuring a correct evolution of both models, and demonstrated how to use these rules for a particular (co-)evolution step of the PPU.
Sinem Getir is a doctoral student at Humboldt University Berlin, Germany. She has research interests lie in the field of software engineering upon the reliable software systems, probabilistic and incremental verification, model driven development and of quality assurance of evolving software systems.
References (91)
- et al.
Identifying critical architectural components with spectral analysis of fault trees
Appl. Soft Comput.
(2016) - et al.
Automatic synthesis of dynamic fault trees from UML systemmodels
Proceedings of the Thirteenth International Symposium on Software Reliability Engineering (ISSRE)
(2002) - et al.
Consistency-preserving edit scripts in model versioning
Proceedings of the Twenty-Eighth IEEE/ACM International Conference on Automated Software Engineering (ASE)
(2013) - et al.
An interactive tool for UML class model evolution in database applications
Softw. Syst. Model.
(2015) - et al.
A system dependability modeling framework using AADL and GSPNs
Proceedings of the 2007 Architecting Dependable Systems IV
(2007) - et al.
Automatic fault tree generation as a support for safety studies of railway interlocking systems
Proceedings of the IFAC Symposium on Control in Transportation Systems
(2009) - et al.
Determining configuration probabilities of safety-critical adaptive systems
Proceedings of the Twenty-First International Conference on Advanced Information Networking and Applications (AINA 2007)
(2007) - Amari, S., Dill, G., Howald, E., 2003. A new approach to solve dynamic fault trees. Proceedings of the 2003 Annual...
- et al.
Henshin: advanced concepts and tools for in-place EMF model transformations
Proceedings of the Thirteenth International Conference on Model Driven Engineering on Languages and Systems, MODELS 2010, Part I
(2010) - et al.
Basic concepts and taxonomy of dependable and secure computing
IEEE Trans. Dependable Secur. Comput.
(2004)
Dynamic fault-tree models for fault-tolerant computer systems
IEEE Trans. Reliab.
Change-driven model transformations
Softw. Syst. Model.
Automated dependability analysis of UML designs
Proceedings of the 1999 IEEE International Symposium on Object-Oriented Real-time distributed Computing
The compass approach: correctness, modelling and performability of aerospace systems
Safety, dependability and performance analysis of extended AADL models
Comput. J.
Model-driven software engineering in practice
Synthesis Lectures on Software Engineering
Model-based safety analysis of a flap control system
Proceedings of the Fourteenth Annual INCOSE Symposium
JTL: a bidirectional and change propagating transformation language
Proceedings of the Third International Conference on Software Language Engineering, SLE 2010
Failure mode and effect analysis for safety-critical systems with software components
Formal support for fault modelling and analysis
Towards a better interaction between design and dependability analysis: FMEA derived from UML/sysML models
Analyzing dynamic fault trees derived from model-based system architectures
Nucl. Eng. Technol. Int. J. Korean Nucl. Soc.
Supporting the co-evolution of metamodels and constraints through incremental constraint management
Proceedings of the 2013 International Conference on Model Driven Engineering Languages and Systems
Integrating safety analyses and component-based design
Proceedings of the Twentieth International Conference on Computer Safety, Reliability and Security, SAFECOMP 2008
Generating and evaluating choices for fixing inconsistencies in UML design models
Proceedings of the Twenty-Third IEEE/ACM International Conference on Automated Software Engineering (ASE 2008)
Safety-oriented design of component assemblies using safety interfaces
Proceedings of the 2007 Formal Aspects of Component Software
Inconsistency handling in multiperspective specifications
IEEE Trans. Softw. Eng.
Cowolf – a generic framework for multi-view co-evolution and evaluation of models
Proceedings of the Eighth International Conference on Theory and Practice of Model Transformations, ICMT 2015, Held as Part of STAF 2015
Co-evolution of software architecture and fault tree models: an explorative case study on a pick and place factory automation system
Proceedings of the Fifth International Workshop Non-functional Properties in Modeling: Analysis, Languages and Processes Co-located with Sixteenth International Conference on Model Driven Engineering Languages and Systems (MODELS 2013 (NIM-ALP 2013)
Component-based hazard analysis: optimal designs, product lines, and online-reconfiguration
Proceedings of the SAFECOMP 200
Compositional hazard analysis of UML component and deployment models
Design for future: managed software evolution
Comput. Sci.-Res. Dev.
Comparing relational model transformation technologies: implementing query/view/transformation with triple graph grammars
Softw. Syst. Model.
Towards an integration of standard component-based safety evaluation techniques with save CCM
Probabilistic model-checking support for FMEA
Proceedings of the Fourth International Conference on the Quantitative Evaluaiton of Systems (QEST 2007)
A comparative study into architecture-based safety evaluation methodologies using AADL’S Error annex and failure propagation models
Proceedings of the Eleventh IEEE High Assurance Systems Engineering Symposium, HASE 2008
Automatic generation of analyzable failure propagation models from component-level failure annotations
Proceedings of the Fifth International Conference on Quality Software (QSIC 2005)
Model-driven safety evaluation with state-event-based component failure annotations
Proceedings of the Eighth International Symposium on Component-Based Software Engineering, CBSE 2005
A framework for qualitative and quantitative formal model-based safety analysis
Proceedings of the Twelfth IEEE High Assurance Systems Engineering Symposium, HASE 2010
Using deductive cause-sequence analysis (DCCA) with SCADE
Proceedings of the SAFECOMP 2007
Deviation analysis: a new use of model checking
Autom. Softw. Eng.
Cited by (18)
Mining domain-specific edit operations from model repositories with applications to semantic lifting of model differences and change profiling
2023, Automated Software EngineeringA Transformational Approach to Managing Data Model Evolution of Web Services
2023, IEEE Transactions on Services ComputingSystem architecture synthesis for performability by logic solvers
2022, Proceedings - 25th ACM/IEEE International Conference on Model Driven Engineering Languages and Systems, MODELS 2022Challenges with Providing Reliability Assurance for Self-Adaptive Cyber-Physical Systems
2022, 2022 6th International Conference on System Reliability and Safety, ICSRS 2022Characteristics, potentials, and limitations of open-source Simulink projects for empirical research
2021, Software and Systems Modeling
Sinem Getir is a doctoral student at Humboldt University Berlin, Germany. She has research interests lie in the field of software engineering upon the reliable software systems, probabilistic and incremental verification, model driven development and of quality assurance of evolving software systems.
Lars Grunske is currently Professor at the Department of Computer Science at Humboldt University Berlin, Germany. He has active research interests in the areas modelling and verification of systems and software. His main focus is on automated analysis, mainly probabilistic and timed model checking and model-based dependability evaluation of complex software intensive systems.
André van Hoorn is a member of the Reliable Software Systems (RSS) group at the University of Stuttgart, Germany. His research interests are in the area of architecture-based software performance engineering and software reengineering. Particularly, he is interested in application performance monitoring, modelling, and management, as well as dynamic/hybrid analysis of legacy systems for architecture recovery and evolution.
Timo Kehrer is heading the Model-Driven Software Engineering Group at the Department of Computer Science at Humboldt-University of Berlin, Germany. He has active research interests in various areas of model-based software and system development with a particular focus on model evolution.
Yannic Noller is a doctoral student at Humboldt University Berlin, Germany. His research interests includes automated software verification, probabilistic software analysis and software architecture. He has a M.Sc. degree from the University of Stuttgart.
Matthias Tichy is currently Professor at the Institute of Software Engineering and Programming Languages at University of Ulm, Germany. His main research focus is on domain specific languages, particularly for mechatronic systems, graph transformations as underlying formal basis in many of these research activities. Furthermore, he employs empirical research methods to understand how humans use software as well as to evaluate technical contributions.