Skip to main content
SpringerLink
Log in

Developing and evolving a DSL-based approach for runtime monitoring of systems of systems

  • Published:
Automated Software Engineering Aims and scope Submit manuscript

Abstract

Complex software-intensive systems are often described as systems of systems (SoS) due to their heterogeneous architectural elements. As SoS behavior is often only understandable during operation, runtime monitoring is needed to detect deviations from requirements. Today, while diverse monitoring approaches exist, most do not provide what is needed to monitor SoS, e.g., support for dynamically defining and deploying diverse checks across multiple systems. In this paper we report on our experiences of developing, applying, and evolving an approach for monitoring an SoS in the domain of industrial automation software, that is based on a domain-specific language (DSL). We first describe our initial approach to dynamically define and check constraints in SoS at runtime and then motivate and describe its evolution based on requirements elicited in an industry collaboration project. We furthermore describe solutions we have developed to support the evolution of our approach, i.e., a code generation approach and a framework to automate testing the DSL after changes. We evaluate the expressiveness and scalability of our new DSL-based approach using an industrial SoS. We also discuss lessons we learned. Our results show that while developing a DSL-based approach is a good solution to support industrial users, one must prepare the approach for evolution, by making it extensible and adaptable to future scenarios. Particularly, support for automated (re-)generation of tools and code after changes and automated testing are essential.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

References

  • Aktug, I., Dam, M., Gurov, D.: Provably correct runtime monitoring. In: Formal Methods (FM’08), pp. 262–277. Springer (2008)

  • Albuquerque, D., Cafeo, B., Garcia, A., Barbosa, S., Abrahão, S., Ribeiro, A.: Quantifying usability of domain-specific languages: an empirical study on software maintenance. J. Syst. Softw. 101, 245–259 (2015)

    Article  Google Scholar 

  • Autili, M., Grunske, L., Lumpe, M., Pelliccione, P., Tang, A.: Aligning qualitative, real-time, and probabilistic property specification patterns using a structured English grammar. IEEE Trans. Softw. Eng. 41(7), 620–638 (2015)

    Article  Google Scholar 

  • Baresi, L., Guinea, S.: Event-based multi-level service monitoring. In: Proceedings of the 20th International Conference on Web Services, pp. 83–90. IEEE (2013)

  • Bauer, A., Leucker, M., Schallhart, C.: Monitoring of real-time properties. In: Foundations of Software Technology and Theoretical Computer Science, pp. 260–272. Springer (2006)

  • Bertolino, A., Calabrò, A., Lonetti, F., Di Marco, A., Sabetta, A.: Towards a model-driven infrastructure for runtime monitoring. In: Software Engineering for Resilient Systems, pp. 130–144. Springer (2011)

  • Bubak, M., Funika, W., Smetek, M., Kiliański, Z., Wismüller, R.: Event handling in the J-OCM monitoring system. In: Parallel Processing and Applied Mathematics, pp. 344–351. Springer (2004)

  • Bures, T., Hnetynka, P., Plasil, F.: Strengthening architectures of smart CPS by modeling them as runtime product-lines. In: Proceedings of the 17th International ACM SIGSOFT Symposium on Component-Based Software Engineering, pp. 91–96. ACM (2014)

  • Cailliau, A., van Lamsweerde, A.: Runtime monitoring and resolution of probabilistic obstacles to system goals. In: 2017 IEEE/ACM 12th International Symposium on Software Engineering for Adaptive and Self-Managing Systems, pp. 1–11. IEEE (2017)

  • Calinescu, R., Ghezzi, C., Kwiatkowska, M.Z., Mirandola, R.: Self-adaptive software needs quantitative verification at runtime. Commun. ACM 55(9), 69–77 (2012)

    Article  Google Scholar 

  • Cazzola, W., Poletti, D.: DSL evolution through composition. In: Proceedings of the 7th Workshop on Reflection, AOP and Meta-Data for Software Evolution, pp. 6:1–6:6. ACM (2010)

  • Chen, F., d’Amorim, M., Roşu, G.: A formal monitoring-based framework for software development and analysis. In: Formal Methods and Software Engineering, pp. 357–372. Springer (2004)

  • De Geest, G., Vermolen, S., Van Deursen, A., Visser, E.: Generating version convertors for domain-specific languages. In: Proceedings of the 15th Working Conference on Reverse Engineering, pp. 197–201. IEEE (2008)

  • Degueule, T., Combemale, B., Blouin, A., Barais, O.: Reusing legacy DSLs with melange. In: Proceedings of the Workshop on Domain-Specific Modeling, pp. 45–46. ACM (2015)

  • Delgado, N., Gates, A.Q., Roach, S.: A taxonomy and catalog of runtime software-fault monitoring tools. IEEE Trans. Softw. Eng. 30(12), 859–872 (2004)

    Article  Google Scholar 

  • Dwyer, M.B., Avrunin, G.S., Corbett, J.C.: Patterns in property specifications for finite-state verification. In: Proceedings of the International Conference on Software Engineering, pp. 411–420. IEEE (1999)

  • Egyed, A.: Instant consistency checking for the UML. In: Proceedings of the 28th International Conference on Software Engineering, pp. 381–390. ACM (2006)

  • Eichelberger, H., Schmid, K.: Flexible resource monitoring of Java programs. J. Syst. Softw. 93, 163–186 (2014)

    Article  Google Scholar 

  • Fickas, S., Feather, M.S.: Requirements monitoring in dynamic environments. In: Proceedings of the 2nd IEEE International Symposium on Requirements Engineering, pp. 140–147. IEEE (1995)

  • Ghezzi, C., Mocci, A., Sangiorgio, M.: Runtime monitoring of component changes with Spy@Runtime. In: 34th International Conference on Software Engineering, pp. 1403–1406. IEEE (2012)

  • Gunadi, H., Tiu, A.: Efficient runtime monitoring with metric temporal logic: a case study in the Android operating system. In: Proceedings Formal Methods 2014, pp. 296–311. Springer (2014)

  • Hermans, F., Pinzger, M., van Deursen, A.: Domain-specific languages in practice: a user study on the success factors. In: Schürr, A., Selic, B. (eds.) Model Driven Engineering Languages and Systems, pp. 423–437. Springer, Berlin (2009)

    Chapter  Google Scholar 

  • Izquierdo, J.L.C., Cabot, J.: Community-driven language development. In: Proceedings of the 2012 4th International Workshop on Modeling in Software Engineering, pp. 29–35 (2012)

  • Juergens, E., Pizka, M.: The language evolver lever—tool demonstration. Electr. Notes Theor. Comput. Sci. 164(2), 55–60 (2006)

    Article  Google Scholar 

  • Kanstrén, T.: A systematic review and taxonomy of runtime invariance in software behaviour. Int. J. Adv. Softw. 4(3 and 4), 256–274 (2011)

    Google Scholar 

  • Kim, M., Viswanathan, M., Kannan, S., Lee, I., Sokolsky, O.: Java-MaC: a run-time assurance approach for Java programs. Form. Methods Syst. Des. 24(2), 129–155 (2004)

    Article  Google Scholar 

  • Kiviluoma, K., Koskinen, J., Mikkonen, T.: Run-time monitoring of architecturally significant behaviors using behavioral profiles and aspects. In: Proceedings of the 2006 International Symposium on Software Testing and Analysis, pp. 181–190. ACM (2006)

  • Krismayer, T., Rabiser, R., Grünbacher, P.: Mining constraints for event-based monitoring in systems of systems. In: Proceedings of the 32nd IEEE/ACM International Conference on Automated Software Engineering, pp. 826–831. IEEE (2017)

  • Luckham, D.C.: Event Processing for Business: Organizing the Real-Time Enterprise. Wiley, New York (2011)

    Google Scholar 

  • Ludwig, T., Wismueller, R., Sundcram, V., Bode, A.: OMIS: On-line Monitoring Interface Specification (v. 2.0). Technical report TUM-I9733, Technische Universität München (1997)

  • Maiden, N.: Monitoring our requirements. IEEE Softw. 30(1), 16–17 (2013)

    Article  Google Scholar 

  • Maier, M.W.: Architecting principles for systems-of-systems. Syst. Eng. 1(4), 267–284 (1998)

    Article  Google Scholar 

  • Mansouri-Samani, M., Sloman, M.: Monitoring distributed systems. IEEE Netw. 7(6), 20–30 (1993)

    Article  Google Scholar 

  • Meyers, B., Vangheluwe, H.: A framework for evolution of modelling languages. Sci. Comput. Program. 76(12), 1223–1246 (2011)

    Article  Google Scholar 

  • Montali, M., Maggi, F.M., Chesani, F., Mello, P., van der Aalst, W.M.: Monitoring business constraints with the event calculus. ACM Trans. Intell. Syst. Technol. 5(1), 17:1–17:30 (2014)

    Google Scholar 

  • Muccini, H., Polini, A., Ricci, F., Bertolino, A.: Monitoring architectural properties in dynamic component-based systems. In: Component-Based Software Engineering, LNCS 4608, pp. 124–139. Springer (2007)

  • Nielsen, C.B., Larsen, P.G., Fitzgerald, J., Woodcock, J., Peleska, J.: Systems of systems engineering: basic concepts, model-based techniques, and research directions. ACM Comput. Surv. 48(2), 18:1–18:41 (2015)

    Article  Google Scholar 

  • Nikolov, N., Rossini, A., Kritikos, K.: Integration of DSLs and migration of models: a case study in the cloud computing domain. Procedia Comput. Sci. 68, 53–66 (2015)

    Article  Google Scholar 

  • Paschke, A.: RBSLA—a declarative rule-based service level agreement language based on RuleML. In: International Conference on Computational Intelligence for Modelling, Control and Automation and International Conference on Intelligent Agents, Web Technologies and Internet Commerce, vol. 2, pp. 308–314. IEEE (2005)

  • Pizka, M., Jurgens, E.: Automating language evolution. In: Proceedings of the 1st Joint IEEE/IFIP Symposium on Theoretical Aspects of Software Engineering, pp. 305–315. IEEE (2007)

  • Rabiser, R., Vierhauser, M., Grünbacher, P.: Assessing the usefulness of a requirements monitoring tool: a study involving industrial software engineers. In: Proceedings of the 38th International Conference on Software Engineering, Companion, pp. 122–131. ACM (2016)

  • Rabiser, R., Vierhauser, M., Grünbacher, P.: Variability management for a runtime monitoring infrastructure. In: Proceedings of the 9th International Workshop on Variability Modelling of Software-Intensive Systems, pp. 35–42. ACM (2015)

  • Rabiser, R., Guinea, S., Vierhauser, M., Baresi, L., Grünbacher, P.: A comparison framework for runtime monitoring approaches. J. Syst. Softw. 125(March), 309–321 (2017)

    Article  Google Scholar 

  • Robinson, W.N.: A requirements monitoring framework for enterprise systems. Requir. Eng. 11(1), 17–41 (2006)

    Article  Google Scholar 

  • Robinson, W.N.: Extended OCL for goal monitoring. Electron. Commun. Eur. Assoc. Softw. Sci. Technol. 9(1), 1–12 (2008)

    MathSciNet  Google Scholar 

  • Sammapun, U., Lee, I., Sokolsky, O.: RT-MaC: runtime monitoring and checking of quantitative and probabilistic properties. In: Proceedings of the 11th IEEE International Conference on Embedded and Real-Time Computing Systems and Applications, pp. 147–153. IEEE (2005)

  • Schmidt, M., Wider, A., Scheidgen, M., Fischer, J., von Klinski, S.: Refactorings in language development with asymmetric bidirectional model transformations. In: Proceedings of the 16th International SDL Forum on Model-Driven Dependability Engineering, pp. 222–238. Springer (2013)

  • Skene, J., Emmerich, W.: Engineering runtime requirements-monitoring systems using MDA technologies. In: Nicola, R.D., Sangiorgi, D. (eds.) Trustworthy Global Computing, pp. 319–333. Springer (2005). https://doi.org/10.1007/11580850

    MATH  Google Scholar 

  • Spanoudakis, G., Kloukinas, C., Mahbub, K.: The SERENITY runtime monitoring framework. In: Kokolakis, S., Gómez, A.M., Spanoudakis, G. (eds.) Security and Dependability for Ambient Intelligence, pp. 213–237. Springer (2009). https://doi.org/10.1007/978-0-387-88775-3

    Google Scholar 

  • Spinellis, D.: Notable design patterns for domain-specific languages. J. Syst. Softw. 56(1), 91–99 (2001)

    Article  Google Scholar 

  • Thanhofer-Pilisch, J., Lang, A., Vierhauser, M., Rabiser, R.: A systematic mapping study on DSL evolution. In: Proceedings of the 43rd Euromicro Conference on Software Engineering and Advanced Applications, pp. 149–156. IEEE (2017)

  • van den Bos, J., van der Storm, T.: A case study in evidence-based DSL evolution. In: Proceedings of the 9th European Conference on Modelling Foundations and Applications, pp. 207–219. Springer (2013)

  • Van Deursen, A., Klint, P., Visser, J.: Domain-specific languages: an annotated bibliography. ACM SIGPLAN Not. 35(6), 26–36 (2000)

    Article  Google Scholar 

  • van Hoorn, A., Waller, J., Hasselbring, W.: Kieker: A framework for application performance monitoring and dynamic software analysis. In: Proceedings of the 3rd Joint International Conference on Performance Engineering, pp. 247–248. ACM (2012)

  • van Lamsweerde, A.: Requirements Engineering: From System Goals to UML Models to Software Specifications. Wiley, New York (2009)

    Google Scholar 

  • Vermolen, S., Visser, E.: Heterogeneous coupled evolution of software languages. In: Proceedings of the 11th International Conference on Model Driven Engineering Languages and Systems, pp. 630–644. Springer (2008)

  • Vierhauser, M., Grünbacher, P., Egyed, A., Rabiser, R., Heider, W.: Flexible and scalable consistency checking on product line variability models. In: Proceedings of the International Conference on Automated Software Engineering, pp. 63–72. ACM (2010)

  • Vierhauser, M., Grünbacher, P., Heider, W., Holl, G., Lettner, D.: Applying a consistency checking framework for heterogeneous models and artifacts in industrial product lines. In: Model Driven Engineering Languages and Systems, pp. 531–545. Springer (2012)

  • Vierhauser, M., Rabiser, R., Cleland-Huang, J.: From requirements monitoring to diagnosis support in system of systems. In: Proceedings of the 23rd International Working Conference on Requirements Engineering: Foundation for Software Quality, pp. 181–187. Springer (2017)

  • Vierhauser, M., Rabiser, R., Grünbacher, P., Egyed, A.: Developing a DSL-based approach for event-based monitoring of systems of systems: experiences and lessons learned. In: Proceedings of the 30th IEEE/ACM International Conference on Automated Software Engineering, pp. 715–725. IEEE (2015)

  • Vierhauser, M., Rabiser, R., Grünbacher, P.: A case study on testing, commissioning, and operation of very-large-scale software systems. In: Proceedings of the 36th International Conference on Software Engineering, Companion, pp. 125–134. ACM (2012)

  • Vierhauser, M., Rabiser, R., Grünbacher, P.: Requirements monitoring frameworks: a systematic review. Inf. Softw. Technol. 80, 89–109 (2016a)

    Article  Google Scholar 

  • Vierhauser, M., Rabiser, R., Grünbacher, P., Seyerlehner, K., Wallner, S., Zeisel, H.: ReMinds: a flexible runtime monitoring framework for systems of systems. J. Syst. Softw. 112, 123–136 (2016b)

    Article  Google Scholar 

  • Viswanathan, M., Kim, M.: Foundations for the run-time monitoring of reactive systems—fundamentals of the MaC language. In: Liu, Z., Araki, K. (eds.) Theoretical Aspects of Computing, 1st International ICTAC Colloquium, (Revised Selected Papers), Lecture Notes in Computer Science 3407, pp. 543–556. Springer (2005)

  • Voelter, M., Visser, E.: Product line engineering using domain-specific languages. In: Proceedings of the 15th International Software Product Line Conference, IEEE CS, pp. 70–79 (2011)

  • Völz, M., Koldehofe, B., Rothermel, K.: Supporting strong reliability for distributed complex event processing systems. In: Proceedings of the 13th International Conference on High Performance Computing & Communication, pp. 477–486. IEEE (2011)

  • Whittle, J., Sawyer, P., Bencomo, N., Cheng, B., Bruel, J.M.: RELAX: a language to address uncertainty in self-adaptive systems requirement. Requir. Eng. 15(2), 177–196 (2010)

    Article  Google Scholar 

  • Wieringa, R., Maiden, N., Mead, N., Rolland, C.: Requirements engineering paper classification and evaluation criteria: a proposal and a discussion. Requir. Eng. 11(1), 102–107 (2006)

    Article  Google Scholar 

  • Zhang, P., Li, B., Muccini, H., Sun, M.: An approach to monitor scenario-based temporal properties in Web service compositions. In: Advanced Web and Network Technologies, and Applications, pp. 144–154. Springer (2008)

  • Zhang, P., Li, W., Wan, D., Grunske, L.: Monitoring of probabilistic timed property sequence charts. Softw. Pract. Exp. 41(7), 841–866 (2011)

    Article  Google Scholar 

  • Zschaler, S., Kolovos, D.S., Drivalos, N., Paige, R.F., Rashid, A.: Domain-specific metamodelling languages for software language engineering. In: Proceedings of the 2nd International Conference on Software Language Engineering, pp. 334–353. Springer (2009)

Download references

Acknowledgements

This work has been supported by the Christian Doppler Forschungsgesellschaft (Grant No. CDL MEVSS), Austria and Primetals Technologies. Michael Vierhauser’s work was funded by the Austrian Science Fund (FWF) under Grant No. J3998-N31. We want to thank Thomas Krismayer for his feedback and his help in setting up the evaluation environment.

Author information

Authors and Affiliations

  1. Christian Doppler Lab. MEVSS, Johannes Kepler University Linz, Linz, Austria

    Rick Rabiser & Jürgen Thanhofer-Pilisch

  2. Computer Science and Engineering, University of Notre Dame, Notre Dame, IN, USA

    Michael Vierhauser

  3. Institute for Software Systems Engineering, Johannes Kepler University Linz, Linz, Austria

    Paul Grünbacher & Alexander Egyed

Authors
  1. Rick Rabiser
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jürgen Thanhofer-Pilisch
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Michael Vierhauser
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Paul Grünbacher
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Alexander Egyed
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Rick Rabiser.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Appendix: grammar of the CDSL v.2

Appendix: grammar of the CDSL v.2

figure e

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Rabiser, R., Thanhofer-Pilisch, J., Vierhauser, M. et al. Developing and evolving a DSL-based approach for runtime monitoring of systems of systems. Autom Softw Eng 25, 875–915 (2018). https://doi.org/10.1007/s10515-018-0241-x

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10515-018-0241-x

Keywords

Search

Navigation