Skip to main content
SpringerLink
Log in

Model-driven performance prediction of systems of systems

  • Theme Section Paper
  • Published:
Software & Systems Modeling Aims and scope Submit manuscript

Abstract

Systems of systems exhibit characteristics that pose difficulty in modelling and predicting their overall performance capabilities, including the presence of operational independence, emergent behaviour, and evolutionary development. When considering systems of systems within the autonomous defence systems context, these aspects become increasingly critical, as constraints on the performance of the final system are typically driven by hard constraints on space, weight and power. System execution modelling languages and tools permit early prediction of the performance of model-driven systems; however, the focus to date has been on understanding the performance of a model rather than determining whether it meets performance requirements, and only subsequently carrying out analysis to reveal the causes of any requirement violations. Moreover, such an analysis is even more difficult when applied to several systems cooperating to achieve a common goal—a system of systems. In this article, we propose an integrated approach to performance prediction of model-driven real-time embedded defence systems and systems of systems. Our architectural prototyping system supports a scenario-driven experimental platform for evaluating model suitability within a set of deployment and real-time performance constraints. We present an overview of our performance prediction system, demonstrating the integration of modelling, execution and performance analysis, and discuss a case study to illustrate our approach.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16

Similar content being viewed by others

Notes

  1. https://www.tareasos.eu/.

  2. http://www.danse-ip.eu/home/.

  3. http://www.compass-research.eu/.

  4. http://amadeos-project.eu.

  5. http://local4global-fp7.eu/.

  6. http://www.cpsos.eu.

  7. http://graphml.graphdrawing.org.

References

  1. Baker, P., Loh, S., Weil, F.: Model-driven engineering in a large industrial context - motorola case study. In: Briand, L., Williams, C. (eds.) Model Driven Engineering Languages and Systems. LNCS, vol. 3713, pp. 476–491. Springer, Berlin (2005)

  2. Balasubramanian, K., Balasubramanian, J., Parsons, J., Gokhale, A., Schmidt, D.C.: Component modeling language for distributed real-time and embedded systems. In: Proceedings of the 11th IEEE Real Time on Embedded Technology and Applications Symposium (RTAS 05), pp. 190–199 (2005)

  3. Balsamo, S., Marco, A.D., Inverardi, P., Simeoni, M.: Model-based performance prediction in software development: a survey. IEEE Trans. Softw. Eng. 30, 295–310 (2004)

    Article  Google Scholar 

  4. Barbierato, E., Gribaudo, M., Iacono, M.: Defining formalisms for performance evaluation with SIMTHESys. Electron. Notes Theor. Comput. Sci. 275, 37–51 (2011). doi:10.1016/j.entcs.2011.09.004. http://www.sciencedirect.com/science/article/pii/S1571066111000958

  5. Beydeda, S., Book, M., Gruhn, V. (eds.): Model Driven Software Development. Spinger, Berlin (2010)

    MATH  Google Scholar 

  6. Boardman, J., Sauser, B.: System of systems—the meaning of OF. In: IEEE/SMC International Conference on System of Systems Engineering (2006). doi:10.1109/SYSOSE.2006.1652284

  7. Boehm, B.: Some future software engineering opportunities and challenges. In: Nanz, S. (ed.) The Future of Software Engineering, pp. 1–32. Springer, Berlin (2011). doi:10.1007/978-3-642-15187-3_1

  8. Bown, R., Sahin, E.: A net-centric xml based system of systems architecture for human tracking. In: Proceedings of Fifth IEEE International Symposium on Service Oriented System Engineering (SoSE’2010), pp. 1–6 (2010)

  9. Bryans, J., Fitzgerald, J., Payne, R., Miyazawa, A., Kristensen, K.: SysML contracts for systems of systems. In: IEEE Systems of Systems Engineering Conference (2014)

  10. Chen, B., Zhang, L., Liu, X., Vangheluwe, H.: Activity-based simulation using devs: increasing performance by an activity model in parallel DEVS simulation. J. Zhejiang Univ. Sci. C 15(1), 13–30 (2014). doi:10.1631/jzus.C1300121

    Article  Google Scholar 

  11. Chiprianov, V., Falkner, K., Szabo, C., Puddy, G.: Architectural support for model-driven performance prediction of distributed real-time embedded systems of systems. In: Proceedings of the 2014 European Conference on Software Architecture. Lecture Notes in Computer Science, vol. 8627, pp. 357–364 (2014)

  12. Cook, S.C.: 2.3.1 on the acquisition of systems of systems. INCOSE Int. Sympos. 11(1), 383–390 (2001). doi:10.1002/j.2334-5837.2001.tb02318.x

  13. Dagli, C.H., Kilicay-Ergin, N.: System of Systems Architecting. Wiley, Hoboken (2008)

    Book  Google Scholar 

  14. Denaro, G., Polini, A., Emmerich, W.: Early performance testing of distributed software applications. SIGSOFT Softw. Eng. Notes 29, 94–103 (2004)

    Article  Google Scholar 

  15. Deng, G., Balasubramanian, J., Otte, W., Schmidt, D., Gokhale, A.: Dance: a qos-enabled component deployment and configuration engine. In: Dearle, A., Eisenbach, S. (eds.) Component Deployment. Lecture Notes in Computer Science, vol. 3798, pp. 67–82. Springer. Berlin (2005). doi:10.1007/11590712_6

  16. Edwards, G., Malek, S., Medvidovic, N.: Scenario-driven dynamic analysis of distributed architectures. In: Proceedings of the 10th International Conference on Fundamental Approaches to Software Engineering, pp. 125–139 (2007)

  17. Esponoza, H., Dubois, H., Gerard, S., Medina, J., Petriu, D., Woodside, M.: Annotating uml models with non-functional properties for quantitative analysis. In: Satellite Events at the MoDELS 2005 Conference, pp. 79–90 (2006)

  18. Eusgeld, I., Nan, C., Dietz, S.: System-of-systems approach for interdependent critical infrastructures. Reliab. Eng. Syst. Saf. 96(6), 679–686 (2011). doi:10.1016/j.ress.2010.12.010

    Article  Google Scholar 

  19. Falkner, K., Chiprianov, V., Falkner, N., Szabo, C., Hill, J., Puddy, G., Fraser, D., Johnston, A., Rieckmann, M., Wallis, A.: Model-driven performance prediction of distributed real-time embedded defence systems. In: The 18th International Conference on Engineering of Complex Computer Systems, Singapore, pp. 155–158 (2013)

  20. Falkner, K., Chiprianov, V., Falkner, N., Szabo, C., Puddy, G.: Modeling scenarios for the performance prediction of distributed real-time embedded systems. In: Military Communications and Information Systems Conference, Canberra, Australia, pp. 1–6 (2013)

  21. Farcas, C., Farcas, E., Krueger, I., Menarini, M.: Addressing the integration challenge for avionics and automotive systems: from components to rick services. Proc. IEEE 98(4), 562–583 (2010)

    Article  Google Scholar 

  22. Franceschinis, G., Gribaudo, M., Iacono, M., Marrone, S., Moscato, F., Vittorini, V.: Interfaces and binding in component based development of formal models. ICST (2010). doi:10.4108/ICST.VALUETOOLS2009.7677

  23. Gaonkar, S., Keefe, K., Lamprecht, R., Rozier, E., Kemper, P., Sanders, W.H.: Performance and dependability modeling with Möbius. SIGMETRICS Perform. Eval. Rev. 36(4), 16–21 (2009). doi:10.1145/1530873.1530878

    Article  Google Scholar 

  24. Ge, B., Hipel, K.W., Yang, K., Chen, Y.: A data-centric capability-focused approach for system-of-systems architecture modeling and analysis. Syst. Eng. 16(3), 363–377 (2013). doi:10.1002/sys.21253

    Article  Google Scholar 

  25. Gokhale, A., Balasubramanian, K., Krishna, A.S., Balasubramanian, J., Edwards, G., Deng, G., Turkay, E., Parsons, J., Schmidt, D.C.: Model driven middleware: a new paradigm for developing distributed real-time and embedded systems. Sci. Comput. Program. 73(1), 39–58 (2008). doi:10.1016/j.scico.2008.05.005

    Article  MathSciNet  MATH  Google Scholar 

  26. Gorod, A., Gove, R., Sauser, B., Boardman, J.: System of systems management: a network management approach. In: IEEE International Conference on System of Systems Engineering, pp. 1–5 (2007). doi:10.1109/SYSOSE.2007.4304218

  27. Graciano Neto, V.V., Guessi, M., Oliveira, L.B.R., Oquendo, F., Nakagawa, E.Y.: Investigating the model-driven development for systems-of-systems. In: Proceedings of the 2014 European Conference on Software Architecture Workshops (ECSAW ’14), pp. 22:1–22:8. ACM, New York, NY, USA (2014). doi:10.1145/2642803.2642825

  28. Green, T., Petre, M.: Usability analysis of visual programming environments: a ‘cognitive dimensions’ framework. J. Vis. Lang. Comput. 7(2), 131–174 (1996)

    Article  Google Scholar 

  29. Groenda, H.: Improving performance predictions by accounting for the accuracy of composed performance models. In: Proceedings of the international conference on the quality of software architectures (QoSA), pp. 111–116 (2012)

  30. Happe, J., Koziolek, H., Reussner, R.: Facilitating performance predictions using software components. IEEE Softw. 28, 27–33 (2011)

    Article  Google Scholar 

  31. Hata, Y., Kamazaki, Y., Sawayama, T., Taniguchi, K., Nakajima, H.: A heart pulse monitoring system by air pressure and ultrasonic sensor systems. In: Proceedings of the Second IEEE International Symposium on Service Oriented System Engineering (SoSE ’2007), pp. 1–5 (2007)

  32. Hill, J., Schmidt, D., Edmondson, J., Gokhale, A.: Tools for continuously evaluating distributed system qualities. IEEE Softw. 27(4), 65–71 (2010)

    Article  Google Scholar 

  33. Hill, J., Schmidt, D., Porter, A., Slaby, J.: CiCUTS: combining system execution modeling tools with continuous integration environments. In: David, W.B., Roy, S (eds.) Engineering of Computer Based System, pp. 66–75. IEEE, Washington DC, USA (2008). doi:10.1109/ECBS.2008.20

  34. Hill, J., Schmidt, D., Slaby, J.: Designing software-intensive systems: methods and principles. In: System Execution Modeling Tools for Evaluating the Quality of Service of Enterprise Distributed Real-time and Embedded Systems, pp. 335–371. IGI Global (2008)

  35. Hill, J., Slaby, J., Baker, S., Schmidt, D.: System execution modeling tools to evaluate enterprise distributed real-time and embedded system qos. In: Proceedings of the 12th International Conference on Embedded and Real-Time Computing Systems and Applications (2006)

  36. Ingram, C., Payne, R., Perry, S., Holt, J., Hansen, F., Couto, L.D.: Modelling patterns for systems of systems architectures. In: IEEE International Systems Conference (SysCon2014) (2014)

  37. Jamshidi, M.: System of systems engineering—new challenges for the 21st century. IEEE Aerosp. Electron. Syst. Mag. 23(5), 4–19 (2008). doi:10.1109/MAES.2008.4523909

    Article  Google Scholar 

  38. Jasmine, K., Vasantha, R.: Design based performance prediction of component based software products. World Acad. Sci. Eng. Technol. 30, 266–269 (2007)

    Google Scholar 

  39. Kampert, D., Nazari, S., Sonntag, C.: D4.1: Dymasos Engineering Concept Specification. Technical Report, DYMASOS. Technische Universität Dortmund, Dortmund, Germany (2014). http://www.dymasos.eu/wp-content/uploads/2013/12/D4_1-DYMASOS-Engineering-Concept-Specification.pdf

  40. Kewley, R., Cook, J., Goerger, N., Henderson, D., Teague, E.: Federated simulations for systems of systems integration. In: Proceedings of the 2008 Winter Simulation Conference, pp. 1121–1129 (2008)

  41. Kieburtz, R.B., McKinney, L., Bell, J.M., Hook, J., Kotov, A., Lewis, J., Oliva, D.P., Sheard, T., Smith, I., Walton, L.: A software engineering experiment in software component generation. In: Proceedings of the 18th International Conference on Software engineering (ICSE), pp. 542–552 (1996)

  42. Klein, J., van Vliet, H.: A systematic review of system-of-systems architecture research. In: Proceedings of the 9th International ACM Sigsoft Conference on Quality of Software Architectures, pp. 13–22 (2013). doi:10.1145/2465478.2465490

  43. Koziolek, H.: Performance evaluation of component-based software systems: a survey. Perform. Eval. 67(8), 634–658 (2010)

    Article  Google Scholar 

  44. Maier, M.W.: Architecting principles for systems-of-systems. Syst. Eng. 1(4), 267–284 (1998). doi:10.1002/(SICI)1520-6858(1998)1:4<267:AID-SYS3>3.0.CO;2-D

    Article  Google Scholar 

  45. Manthorpe, W.H.J.: The emerging joint system of systems: a systems engineering challenge and opportunity for APL. John Hopkins APL Tech. Dig. 17(3), 305–310 (1996)

  46. Merola, L.: The COTS software obsolescence threat. In: Fifth International Conference on Commercial-off-the-Shelf (COTS)-Based Software Systems (2006)

  47. Michelson, B.M.: Event-Driven Architecture Overview. Technical Report, Patricia Seybold Group. Bridgewater, MA, USA (2006)

  48. Mustafiz, S., Denil, J., Lúcio, L., Vangheluwe, H.: The ftg + pm framework for multi-paradigm modelling: An automotive case study. In: Proceedings of the 6th International Workshop on Multi-Paradigm Modeling, MPM ’12, pp. 13–18. ACM, New York, NY, USA (2012). doi:10.1145/2508443.2508446

  49. OMG: Data Distribution Service for Real-Time Systems Version 1.2. Object Management Group, Needham, MA, USA (2007)

  50. OMG: UML Profile for MARTE: Modeling and Analysis of Real-Time Embedded Systems v 1.1. Object Management Group, Needham, MA, USA (2011)

  51. Paunov, S., Hill, J., Schmidt, D., Baker, S., Slaby, J.: Domain-specific modeling languages for configuring and evaluating enterprise DRE system quality of service. In: 13th Annual IEEE International Symposium and Workshop on Engineering of Computer Based Systems (2006)

  52. Pérez, J., Díaz, J., Garbajosa, J., Yagüe, A., Gonzalez, E., Lopez-Perea, M.: Large-scale smart grids as system of systems. In: Proceedings of the First International Workshop on Software Engineering for Systems-of-Systems, SESoS ’13, pp. 38–42. ACM, New York, NY, USA (2013). doi:10.1145/2489850.2489858

  53. Pisano, N.: Technical performance measurement earned value, and risk management: an integrated diagnostic tool for program management. In: Defense Acquisition University Acquisition Research Symposium, Ft. Belvoir, VA, USA (1995)

  54. Rieckmann, M., Fraser, D., Chiprianov, V., Szabo, C., Falkner, K.: Demonstration of model-driven performance prediction of distributed real-time embedded systems of systems. In: Proceedings of the European Conference on Software Architecture Workshops, ECSAW ’14, pp. 34:1–34:4. ACM, New York, NY, USA (2014). doi:10.1145/2642803.2642837

  55. Rieckmann, M., Fraser, D., Chiprianov, V., Szabo, C., Falkner, K.: Demonstration of model-driven performance prediction of distributed real-time embedded systems of systems. In: Proceedings of the European Conference on Software Architecture Workshops, pp. 34–40 (2014)

  56. Sage, A.P., Cuppan, C.D.: On the systems engineering and management of systems of systems and federation of systems. Inf. Knowl. Syst. Manag. 2, 325–345 (2001)

    Google Scholar 

  57. Schmidt, D.C., Stal, M., Rohnert, H., Bushmann, F.: Pattern-Oriented Software Architecture: Patterns for Concurrent and Networked Objects, vol. 2. Wiley, Hoboken (2000)

    Google Scholar 

  58. Schneider, J.P., Teodorov, C., Senn, E., Champeau, J.: Towards a dynamic infrastructure for playing with systems of systems. In: Proceedings of the European Conference on Software Architecture Workshops (ECSAW ’14), pp. 31:1–31:4. ACM, New York, NY, USA (2014). doi:10.1145/2642803.2642834

  59. Sharawi, A., Sala-Diakanda, S.N., Dalton, A., Quijada, S., Yousef, N., Rabelo, L., Sepulveda, J.: A distributed simulation approach for modeling and analyzing systems of systems. In: Proceedings of the Winter Simulation Conference, pp. 1028–1035 (2006)

  60. SISO: Military Scenario Sefinition Language (MSDL). siso-std-007-2008. Simulation Interoperability Standards Organization, Orlando, FL, USA (2008)

  61. Smith, C.: Introduction to software performance engineering: origins and outstanding problems. In: Proceedings of the 7th International Conference on Formal Methods for Performance Evaluation, pp. 395–428 (2007)

  62. Szabo, C., Chen, Y.: A model-driven change traceability method in system modeling execution. In: Proceedings of the 22nd Australasian Software Engineering Conference (2013)

  63. Trubiani, C., Meedeniya, I., Cortellessa, V., Aleti, A., Grunske, L.: Model-based performance analysis of software architectures under uncertainty. In: Proceedings of the International Conference on the Quality of Software Architectures (QoSA), pp. 69–78 (2013)

  64. Vangheluwe, H., De Lara, J., Mosterman, P.J.: An introduction to multi-paradigm modelling and simulation. In: Proceedings of the AIS2002 Conference (AI, Simulation and Planning in High Autonomy Systems), Lisboa, Portugal, pp. 9–20 (2002)

  65. Vierhauser, M., Rabiser, R., Grünbacher, P., Danner, C., Wallner, S.: Evolving systems of systems: industrial challenges and research perspectives. In: Proceedings of the First International Workshop on Software Engineering for Systems-of-Systems (SESoS ’13), pp. 1–4. ACM, New York, NY, USA (2013). doi:10.1145/2489850.2489851

  66. Volkert Richard, J.T.S., Yu, J.: A framework for performance prediction during development of systems of systems. Int. J. Syst. Syst. Eng. 3, 76–95 (2012)

    Article  Google Scholar 

  67. Weiderman, N., Bergey, J., Smith, D., Tilley, S.: Approaches to legacy system evolution. Technical Report CMU/SEI-97-TR-014, Software Engineering Institute, Carnegie-Mellon University. Pittsburgh, PA, USA (1997)

  68. Wu, X., McMullen, D., Woodside, M.: Component based performance prediction. In: Proceedings of the 6th ICSE Workshop on Component-Based Software Engineering (2003)

  69. You, G.r., Sun, X., Sun, M., Wang, J., Chen, Y.: Bibliometric and social network analysis of the SOS field. In: 9th International Conference on System of Systems Engineering, pp. 13–18 (2014). doi:10.1109/SYSOSE.2014.6892456

Download references

Author information

Authors and Affiliations

  1. School of Computer Science, University of Adelaide, Adelaide, Australia

    Katrina Falkner, Claudia Szabo, Marianne Rieckmann, Dan Fraser & Cathlyn Aston

  2. LIUPPA, University of Pau and Pays Adour, Pau, France

    Vanea Chiprianov

  3. Department of Defence, Defence Science and Technology Organisation, Canberra, Australia

    Gavin Puddy

Authors
  1. Katrina Falkner
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Claudia Szabo
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Vanea Chiprianov
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Gavin Puddy
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Marianne Rieckmann
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Dan Fraser
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Cathlyn Aston
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Claudia Szabo.

Additional information

Communicated by Dr. Kai Sachs and Catalina Llado.

This work was sponsored by the Defence Science and Technology Organisation, Australia.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Falkner, K., Szabo, C., Chiprianov, V. et al. Model-driven performance prediction of systems of systems. Softw Syst Model 17, 415–441 (2018). https://doi.org/10.1007/s10270-016-0547-8

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10270-016-0547-8

Keywords

Search

Navigation