Elsevier

Journal of Systems and Software

Volume 142, August 2018, Pages 115-135
Journal of Systems and Software

Supporting semi-automatic co-evolution of architecture and fault tree models

https://doi.org/10.1016/j.jss.2018.04.001Get rights and content

Highlights

  • A set of intra and inter-model transformation rules for (co)-evolving models.

  • Quantitative analysis of the evolution scenarios: correlation and mining analysis.

  • Evaluation showing the usefulness and completeness of the sets of transformations.

  • Application of co-evolution rules in a tool environment for model co-evolution.

  • Publicly available metamodels, models and evolution scenarios of an industrial plant.

Abstract

During the whole life-cycle of software-intensive systems in safety-critical domains, system models must consistently co-evolve with quality evaluation models like fault trees. However, performing these co-evolution steps is a cumbersome and often manual task. To understand this problem in detail, we have analyzed the evolution and mined common changes of architecture and fault tree models for a set of evolution scenarios of a part of a factory automation system called Pick and Place Unit. On the other hand, we designed a set of intra- and inter-model transformation rules which fully cover the evolution scenarios of the case study and which offer the potential to semi-automate the co-evolution process. In particular, we validated these rules with respect to completeness and evaluated them by a comparison to typical visual editor operations. Our results show a significant reduction of the amount of required user interactions in order to realize the co-evolution.

Introduction

Quality of service (QoS) attributes such as safety, reliability and performance are crucial for software-intensive systems, e.g., in safety-critical or automation systems. Such systems (e.g. aircraft and automation systems, robotics) are not tolerable with an erroneous design since they are playing fundamental roles in human lives. Therefore quality evaluation during the development of systems from design time to runtime is inevitable. A rigorous quality evaluation is among the key methods for the dependable engineering of such systems. To that end, model-based approaches have been proposed which construct quality evaluation models from system models to gain knowledge about the quality of a system by checking these models against formally specified quality requirements (Grunske and Han, 2008).

In model-based quality evaluation, the consistency of the involved models is of utmost importance. For example, the failures of an architectural component must be adequately considered in an associated fault tree model. While this consistency requirement can be reasonably met for a particular snapshot of a system, quality evaluation models typically become outdated when the system evolves, i.e., quality evaluation models and system models evolve in an inconsistent way. As a consequence, quality evaluation leads to unexpected and highly improper results. An example in the context of hazard analysis of component-based embedded systems is the addition of a new port for a sensor of a component without a corresponding addition of the sensor failures in the relevant fault trees. This will clearly lead to wrong hazard analysis results. Hence, loosely inter-related models such as architectural models and quality evaluation models should consistently evolve in parallel, a phenomenon to which we refer to as (consistent) model co-evolution in the remainder of this paper.

Since loosely inter-related models are typically changed in isolation of each other, one adequate approach to support developers is model synchronization, i.e., the task of adapting a model in response to changes in one of its inter-related counterparts in order to achieve consistent co-evolution. In general, however, achieving this kind of model co-evolution is not a straightforward task (Mens et al., 2005): quality evaluation models cannot be fully generated from system models, and most relations between the elements of the different models are not simple one-to-one correspondences. In other words, achieving consistent co-evolution cannot be fully automated as usually assumed by existing approaches to model synchronization (see Section 2 for a more detailed discussion of related work in this area Getir et al., 2013). At best, developers may be supported by recommending possible synchronization actions, e.g. as in the model-based (co-)evolution framework known as CoWolf (Getir et al., 2015). To achieve consistent co-evolution, CoWolf follows a rule-based approach where incremental model transformations are used to recommend both intra- and inter-model change actions. However, since the adequacy of the recommendations strongly depends on the transformation rules being used by the tool, the evolution problem is shifted to the engineering of proper transformation rules. These should capture evolution and co-evolution steps being considered as useful by the developers using the tool.

In this paper, we tackle this problem of engineering proper transformation rule sets for an important class of models in the context of model-based hazard analysis of software-intensive systems, namely system architecture models and fault tree models. We extend our previous work Getir et al. (2013) on the evolution of the so-called Pick and Place Unit (PPU) (Legat et al., 2013), a case study from the automation engineering domain which is commonly used in the German priority program “Design for Future – Managed Software Evolution” (Goltz et al., 2015). To study co-evolution in terms of the PPU, we created consistent software architecture and fault tree models for all safety-relevant evolution scenarios. The contributions of this paper are:

  • 1.

    A thorough quantitative analysis of the evolution scenarios with respect to the co-evolution of the models, i.e., how changes in one model affect changes in the other model. We show that the models do not co-evolve in a systematic, automatable way and instead expertise of the developer is required to achieve co-evolution. This is a minor contribution which, while not being exceedingly surprising, confirms the findings of previous research in this context.

  • 2.

    The major contribution is a set of model transformation rules for (1) the independent evolution of software architecture and fault tree models and (2) synchronization of one model based on changes in another model ensuring a correct co-evolution of both models. In the evaluation of the rules, we show that the presented set of model transformations is complete, i.e., it supports performing all co-evolutions of the case study scenarios, and improves the task efficiency(cf. Quality in Use, ISO/IEC, 2001) by reducing the amount of required model transformation applications to realize the co-evolution by, on average, 52% compared to visual editing operations and 85% compared to atomic model changes. Additionally, we implemented these rules in the tool CoWolf (Getir et al., 2015) to enable the co-evolution of fault trees and software architecture models.

To enable reproducibility of our results, we make all models for the scenarios, the set of model transformation rules as well as the code for the evaluation publicly available.1

The remainder of this paper is structured as follows. In the next section, we discuss related work in the areas of safety evaluation models and their automatic generation as well as different approaches to achieve consistent co-evolution of inter-related models. Section 3 briefly sketches the used modeling and model transformation languages. Thereafter, we present the results of the quantitative analysis of the co-evolution of the models of the case study in Section 4. Section 5 contains a description of the developed inter- and intra-model transformation rules for architecture and fault tree models. We evaluate this set of model transformation rules in Section 6. Finally, we conclude and present an outlook on planned future work in incremental analysis of quality models and supporting the developer selecting model transformations by analyzing historic developer decisions on co-evolutions. The appendix contains a thorough description of all evolution scenarios and their impact on system architecture and fault tree models.

Section snippets

Related work

The work presented in this paper draws from two separate research areas, namely the work on automatic/semi-automatic generation of safety evaluation models in the architectural design phase and general approaches that aim at keeping dependent models consistent while individual models evolve.

Safety evaluation models and their automatic generation: Several evaluation models have been proposed to facilitate a quantitative safety analysis based on architectural specifications (Grunske and Han, 2008

Background

Our work is based on model-driven software engineering, particularly, modeling software architectures and fault trees as well as specifying model transformations. Therefore, we briefly introduce our two modeling languages including their meta-models as well as the Henshin transformation language (Arendt et al., 2010) which we use to specify model transformations.

Case study on architecture and fault tree co-evolution

The system we studied is an industrial production plant, called Pick and Place Unit (PPU). The PPU is a factory automation system that mimics an industrial robot that moves work pieces (WPs) between different working positions where they are stored or processed. Such systems are an interesting case for evolution since they contain mechanical parts, electrical parts, and software parts. All these parts can be evolved individually or in combination. Additionally, these systems are also typically

Supporting co-evolution

In Section 4, we presented the analysis results and conclude that change actions between system architecture and fault trees are not straightforward. For this reason, developers can only be assisted by a recommender system that offers a set of meaningful transformations, which is rich enough to consistently co-evolve system architecture and fault tree models. Specifications of such transformations can be integrated into a rule-based framework such as the CoWolf tool (Getir et al., 2015)

Evaluation

The quality of a co-evolution framework such as CoWolf strongly depends on the quality of the transformation rules being offered to developers as interactive editing commands. Thus, we investigate two major quality aspects of our set of manually defined transformation rules which must be (i) complete in the sense that every couple (SAi, FTi) can be evolved to (SAip, FTip) in a consistent way, and (ii) helpful in the sense that this evolution can be achieved by a developer with minimal effort,

Conclusion

In this paper, we have thoroughly analyzed the co-evolution of architecture models and fault trees for a factory automation system called Pick and Place Unit as an extension of our previous work (Getir et al., 2013).

As a major contribution, we provided a set of model transformation rules for achieving co-evolution of software architecture and fault tree models ensuring a correct evolution of both models, and demonstrated how to use these rules for a particular (co-)evolution step of the PPU.

Sinem Getir is a doctoral student at Humboldt University Berlin, Germany. She has research interests lie in the field of software engineering upon the reliable software systems, probabilistic and incremental verification, model driven development and of quality assurance of evolving software systems.

References (91)

  • J. Bechta-Dugan et al.

    Dynamic fault-tree models for fault-tolerant computer systems

    IEEE Trans. Reliab.

    (1992)
  • G. Bergmann et al.

    Change-driven model transformations

    Softw. Syst. Model.

    (2012)
  • A. Bondavalli et al.

    Automated dependability analysis of UML designs

    Proceedings of the 1999 IEEE International Symposium on Object-Oriented Real-time distributed Computing

    (1999)
  • Boulanger, J. L., Van Quang, D., 2008. Experiences from a Model-Based Methodology for Embedded Electronic Software in...
  • M. Bozzano et al.

    The compass approach: correctness, modelling and performability of aerospace systems

  • M. Bozzano et al.

    Safety, dependability and performance analysis of extended AADL models

    Comput. J.

    (2011)
  • M. Brambilla et al.

    Model-driven software engineering in practice

    Synthesis Lectures on Software Engineering

    (2012)
  • M. Bretschneider et al.

    Model-based safety analysis of a flap control system

    Proceedings of the Fourteenth Annual INCOSE Symposium

    (2004)
  • CENELEC EN 50126,128,129, 2000. CENELEC (European Committee for Electro-Technical Standardisation): Railway...
  • A. Cicchetti et al.

    JTL: a bidirectional and change propagating transformation language

    Proceedings of the Third International Conference on Software Language Engineering, SLE 2010

    (2010)
  • T. Cichocki et al.

    Failure mode and effect analysis for safety-critical systems with software components

  • T. Cichocki et al.

    Formal support for fault modelling and analysis

  • P. David et al.

    Towards a better interaction between design and dependability analysis: FMEA derived from UML/sysML models

  • J. Dehlinger et al.

    Analyzing dynamic fault trees derived from model-based system architectures

    Nucl. Eng. Technol. Int. J. Korean Nucl. Soc.

    (2008)
  • A. Demuth et al.

    Supporting the co-evolution of metamodels and constraints through incremental constraint management

    Proceedings of the 2013 International Conference on Model Driven Engineering Languages and Systems

    (2013)
  • D. Domis et al.

    Integrating safety analyses and component-based design

    Proceedings of the Twentieth International Conference on Computer Safety, Reliability and Security, SAFECOMP 2008

    (2008)
  • A. Egyed et al.

    Generating and evaluating choices for fixing inconsistencies in UML design models

    Proceedings of the Twenty-Third IEEE/ACM International Conference on Automated Software Engineering (ASE 2008)

    (2008)
  • J. Elmqvist et al.

    Safety-oriented design of component assemblies using safety interfaces

    Proceedings of the 2007 Formal Aspects of Component Software

    (2007)
  • A. Finkelstein et al.

    Inconsistency handling in multiperspective specifications

    IEEE Trans. Softw. Eng.

    (1994)
  • S. Getir, L. Grunske, A. van Hoorn, T. Kehrer,Y. Noller, and M. Tichy, (2018), Supporting Semi-Automatic Co-Evolution...
  • S. Getir et al.

    Cowolf – a generic framework for multi-view co-evolution and evaluation of models

    Proceedings of the Eighth International Conference on Theory and Practice of Model Transformations, ICMT 2015, Held as Part of STAF 2015

    (2015)
  • S. Getir et al.

    Co-evolution of software architecture and fault tree models: an explorative case study on a pick and place factory automation system

    Proceedings of the Fifth International Workshop Non-functional Properties in Modeling: Analysis, Languages and Processes Co-located with Sixteenth International Conference on Model Driven Engineering Languages and Systems (MODELS 2013 (NIM-ALP 2013)

    (2013)
  • H. Giese et al.

    Component-based hazard analysis: optimal designs, product lines, and online-reconfiguration

    Proceedings of the SAFECOMP 200

    (2006)
  • H. Giese et al.

    Compositional hazard analysis of UML component and deployment models

  • U. Goltz et al.

    Design for future: managed software evolution

    Comput. Sci.-Res. Dev.

    (2015)
  • J. Greenyer et al.

    Comparing relational model transformation technologies: implementing query/view/transformation with triple graph grammars

    Softw. Syst. Model.

    (2010)
  • L. Grunske

    Towards an integration of standard component-based safety evaluation techniques with save CCM

  • L. Grunske et al.

    Probabilistic model-checking support for FMEA

    Proceedings of the Fourth International Conference on the Quantitative Evaluaiton of Systems (QEST 2007)

    (2007)
  • L. Grunske et al.

    A comparative study into architecture-based safety evaluation methodologies using AADL’S Error annex and failure propagation models

    Proceedings of the Eleventh IEEE High Assurance Systems Engineering Symposium, HASE 2008

    (2008)
  • L. Grunske et al.

    Automatic generation of analyzable failure propagation models from component-level failure annotations

    Proceedings of the Fifth International Conference on Quality Software (QSIC 2005)

    (2005)
  • L. Grunske et al.

    Model-driven safety evaluation with state-event-based component failure annotations

    Proceedings of the Eighth International Symposium on Component-Based Software Engineering, CBSE 2005

    (2005)
  • M. Güdemann et al.

    A framework for qualitative and quantitative formal model-based safety analysis

    Proceedings of the Twelfth IEEE High Assurance Systems Engineering Symposium, HASE 2010

    (2010)
  • M. Gündemann et al.

    Using deductive cause-sequence analysis (DCCA) with SCADE

    Proceedings of the SAFECOMP 2007

    (2007)
  • M.P.E. Heimdahl et al.

    Deviation analysis: a new use of model checking

    Autom. Softw. Eng.

    (2005)
  • Henshin Trace Metamodel, 2016. http://wiki.eclipse.org/Henshin_Trace_Model, last access:...
  • Sinem Getir is a doctoral student at Humboldt University Berlin, Germany. She has research interests lie in the field of software engineering upon the reliable software systems, probabilistic and incremental verification, model driven development and of quality assurance of evolving software systems.

    Lars Grunske is currently Professor at the Department of Computer Science at Humboldt University Berlin, Germany. He has active research interests in the areas modelling and verification of systems and software. His main focus is on automated analysis, mainly probabilistic and timed model checking and model-based dependability evaluation of complex software intensive systems.

    André van Hoorn is a member of the Reliable Software Systems (RSS) group at the University of Stuttgart, Germany. His research interests are in the area of architecture-based software performance engineering and software reengineering. Particularly, he is interested in application performance monitoring, modelling, and management, as well as dynamic/hybrid analysis of legacy systems for architecture recovery and evolution.

    Timo Kehrer is heading the Model-Driven Software Engineering Group at the Department of Computer Science at Humboldt-University of Berlin, Germany. He has active research interests in various areas of model-based software and system development with a particular focus on model evolution.

    Yannic Noller is a doctoral student at Humboldt University Berlin, Germany. His research interests includes automated software verification, probabilistic software analysis and software architecture. He has a M.Sc. degree from the University of Stuttgart.

    Matthias Tichy is currently Professor at the Institute of Software Engineering and Programming Languages at University of Ulm, Germany. His main research focus is on domain specific languages, particularly for mechatronic systems, graph transformations as underlying formal basis in many of these research activities. Furthermore, he employs empirical research methods to understand how humans use software as well as to evaluate technical contributions.

    View full text