Skip to main content
SpringerLink
Log in

Using Runtime Quantitative Verification to Provide Assurance Evidence for Self-Adaptive Software

Advances, Applications and Research Challenges

  • Conference paper
  • First Online:
Software Engineering for Self-Adaptive Systems III. Assurances

Part of the book series: Lecture Notes in Computer Science ((LNPSE,volume 9640))

Abstract

Providing assurance that self-adaptive software meets its dependability, performance and other quality-of-service (QoS) requirements is a great challenge. Recent approaches to addressing it use formal methods at runtime, to drive the reconfiguration of self-adaptive software in provably correct ways. One approach that shows promise is runtime quantitative verification (RQV), which uses quantitative model checking to reverify the QoS properties of self-adaptive software after environmental, requirement and system changes. This reverification identifies QoS requirement violations and supports the dynamic reconfiguration of the software for recovery from such violations. More importantly, it provides irrefutable assurance evidence that adaptation decisions are correct. In this paper, we survey recent advances in the development of efficient RQV techniques, the application of these techniques within multiple domains and the remaining research challenges.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    A service failed if all instances of one of its functions are unavailable.

References

  1. Alur, R., Henzinger, T.A., Vardi, M.Y.: Theory in practice for system design and verification. ACM SIGLOG News 2(1), 46–51 (2015)

    Google Scholar 

  2. Ardagna, D., Pernici, B.: Adaptive service composition in flexible processes. IEEE Trans. Softw. Eng. 33(6), 369–384 (2007)

    Article  Google Scholar 

  3. Aziz, A., Sanwal, K., Singhal, V., Brayton, R.: Model-checking continuous-time Markov chains. ACM Trans. Comput. Logic 1(1), 162–170 (2000)

    Article  MathSciNet  MATH  Google Scholar 

  4. Baier, C., Katoen, J.-P.: Principles of Model Checking. The MIT Press, Cambridge (2008)

    MATH  Google Scholar 

  5. Bianculli, D., Filieri, A., Ghezzi, C., Mandrioli, D.: A syntactic-semantic approach to incremental verification. CoRR, abs/1304.8034 (2013)

    Google Scholar 

  6. Bianculli, D., Filieri, A., Ghezzi, C., Mandrioli, D.: Syntactic-semantic incrementality for agile verification. Sci. Comput. Program. 97(1), 47–54 (2015)

    Article  Google Scholar 

  7. Calinescu, R.: Emerging techniques for the engineering of self-adaptive high-integrity software. In: Cámara, J., de Lemos, R., Ghezzi, C., Lopes, A. (eds.) Assurances for Self-Adaptive Systems: Principles, Models, and Techniques. LNCS, vol. 7740, pp. 297–310. Springer, Heidelberg (2013). https://doi.org/10.1007/978-3-642-36249-1_11

    Chapter  Google Scholar 

  8. Calinescu, R., Gerasimou, S., Banks, A.: Self-adaptive software with decentralised control loops. In: Egyed, A., Schaefer, I. (eds.) FASE 2015. LNCS, vol. 9033, pp. 235–251. Springer, Heidelberg (2015). https://doi.org/10.1007/978-3-662-46675-9_16

    Google Scholar 

  9. Calinescu, R., Ghezzi, C., Johnson, K., Pezze, M., Rafiq, Y., Tamburrelli, G.: Formal verification with confidence intervals to establish quality of service properties of software systems. IEEE Trans. Reliab. PP(99), 1–19 (2015)

    Google Scholar 

  10. Calinescu, R., Ghezzi, C., Kwiatkowska, M., Mirandola, R.: Self-adaptive software needs quantitative verification at runtime. Commun. ACM 55(9), 69–77 (2012)

    Article  Google Scholar 

  11. Calinescu, R., Grunske, L., Kwiatkowska, M., Mirandola, R., Tamburrelli, G.: Dynamic QoS management and optimization in service-based systems. IEEE Trans. Softw. Eng. 37(3), 387–409 (2011)

    Article  Google Scholar 

  12. Calinescu, R., Johnson, K., Paterson, C.: FACT: a probabilistic model checker for formal verification with confidence intervals. In: 22nd International Conference on Tools and Algorithms for the Construction and Analysis of Systems (TACAS) (2016)

    Google Scholar 

  13. Calinescu, R., Johnson, K., Rafiq, Y.: Using observation ageing to improve Markovian model learning in QoS engineering. In: 2nd ACM/SPEC International Conference on Performance Engineering (ICPE 2011), pp. 505–510 (2011)

    Google Scholar 

  14. Calinescu, R., Johnson, K., Rafiq, Y.: Developing self-verifying service-based systems. In: 28th IEEE/ACM International Conference on Automated Software Engineering (ASE 2013), pp. 734–737 (2013)

    Google Scholar 

  15. Calinescu, R., Johnson, K., Rafiq, Y.: Using continual verification to automate service selection in service-based systems. Technical report YCS-2013-484, Department of Computer Science, University of York (2013). http://www.cs.york.ac.uk/ftpdir/reports/2013/YCS/484/YCS-2013-484.pdf

  16. Calinescu, R., Kikuchi, S.: Formal methods @ runtime. In: Calinescu, R., Jackson, E. (eds.) Monterey Workshop 2010. LNCS, vol. 6662, pp. 122–135. Springer, Heidelberg (2011). https://doi.org/10.1007/978-3-642-21292-5_7

    Chapter  Google Scholar 

  17. Calinescu, R., Kikuchi, S., Johnson, K.: Compositional reverification of probabilistic safety properties for large-scale complex IT systems. In: Calinescu, R., Garlan, D. (eds.) Monterey Workshop 2012. LNCS, vol. 7539, pp. 303–329. Springer, Heidelberg (2012). https://doi.org/10.1007/978-3-642-34059-8_16

    Chapter  Google Scholar 

  18. Calinescu, R., Kwiatkowska, M.: Using quantitative analysis to implement autonomic IT systems. In: 31st IEEE International Conference on Software Engineering (ICSE 2009), pp. 100–110 (2009)

    Google Scholar 

  19. Calinescu, R., Rafiq, Y., Johnson, K., Bakir, M.E.: Adaptive model learning for continual verification of non-functional properties. In: 5th ACM/SPEC International Conference on Performance Engineering (ICPE 2014), pp. 87–98 (2014)

    Google Scholar 

  20. Cámara, J., de Lemos, R.: Evaluation of resilience in self-adaptive systems using probabilistic model-checking. In: 2012 ICSE Workshop on Software Engineering for Adaptive and Self-Managing Systems (SEAMS), pp. 53–62, June 2012

    Google Scholar 

  21. Cámara, J., Garlan, D., Schmerl, B., Pandey, A.: Optimal planning for architecture-based self-adaptation via model checking of stochastic games. In: Proceedings of the 10th DADS Track of the 30th ACM Symposium on Applied Computing, Salamanca, Spain, 13–17 April 2015

    Google Scholar 

  22. Cámara, J., Moreno, G.A., Garlan, D.: Stochastic game analysis and latency awareness for proactive self-adaptation. In: Proceedings of the 9th International Symposium on Software Engineering for Adaptive and Self-Managing Systems, SEAMS 2014, pp. 155–164. ACM, New York (2014)

    Google Scholar 

  23. Cardellini, V., Casalicchio, E., Grassi, V., Iannucci, S., Lo Presti, F., Mirandola, R.: Moses: a framework for QoS driven runtime adaptation of service-oriented systems. IEEE Trans. Softw. Eng. 38(5), 1138–1159 (2012)

    Article  Google Scholar 

  24. Cheung, R.: A user-oriented software reliability model. IEEE Trans. Softw. Eng. 6(2), 118–125 (1980)

    Article  MATH  Google Scholar 

  25. Coppit, D., Yang, J., Khurshid, S., Le, W., Sullivan, K.: Software assurance by bounded exhaustive testing. IEEE Trans. Softw. Eng. 31(4), 328–339 (2005)

    Article  Google Scholar 

  26. Daws, C.: Symbolic and parametric model checking of discrete-time Markov chains. In: Liu, Z., Araki, K. (eds.) ICTAC 2004. LNCS, vol. 3407, pp. 280–294. Springer, Heidelberg (2005). https://doi.org/10.1007/978-3-540-31862-0_21

    Chapter  Google Scholar 

  27. de Lemos, R., et al.: Software engineering for self-adaptive systems: a second research roadmap. In: de Lemos, R., Giese, H., Müller, H.A., Shaw, M. (eds.) Software Engineering for Self-Adaptive Systems II. LNCS, vol. 7475, pp. 1–32. Springer, Heidelberg (2013). https://doi.org/10.1007/978-3-642-35813-5_1

    Chapter  Google Scholar 

  28. D’Ippolito, N., Braberman, V., Kramer, J., Magee, J., Sykes, D., Uchitel, S.: Hope for the best, prepare for the worst: multi-tier control for adaptive systems. In: Proceedings of the 36th International Conference on Software Engineering, ICSE 2014, pp. 688–699. ACM, New York (2014)

    Google Scholar 

  29. D’Ippolito, N., Braberman, V., Piterman, N., Uchitel, S.: Synthesis of live behaviour models for fallible domains. In: 33rd International Conference on Software Engineering (ICSE), pp. 211–220, May 2011

    Google Scholar 

  30. D’Ippolito, N.R., Braberman, V., Piterman, N., Uchitel, S.: Synthesis of live behaviour models. In: Proceedings of the Eighteenth ACM SIGSOFT International Symposium on Foundations of Software Engineering, FSE 2010, pp. 77–86. ACM, New York (2010)

    Google Scholar 

  31. Dräger, K., Forejt, V., Kwiatkowska, M., Parker, D., Ujma, M.: Permissive controller synthesis for probabilistic systems. In: Ábrahám, E., Havelund, K. (eds.) TACAS 2014. LNCS, vol. 8413, pp. 531–546. Springer, Heidelberg (2014). https://doi.org/10.1007/978-3-642-54862-8_44

    Chapter  Google Scholar 

  32. Dubslaff, C., Klüppelholz, S., Baier, C.: Probabilistic model checking for energy analysis in software product lines. In: Proceedings of the 13th International Conference on Modularity, MODULARITY 2014, pp. 169–180. ACM, New York (2014)

    Google Scholar 

  33. Epifani, I., Ghezzi, C., Mirandola, R., Tamburrelli, G.: Model evolution by run-time parameter adaptation. In: 31st IEEE International Conference on Software Engineering (ICSE 2009), pp. 111–121 (2009)

    Google Scholar 

  34. Filieri, A., Ghezzi, C.: Further steps towards efficient runtime verification: handling probabilistic cost models. In: Formal Methods Software Engineering: Rigorous and Agile Approaches (FormSERA 2012), pp. 2–8 (2012)

    Google Scholar 

  35. Filieri, A., Ghezzi, C., Tamburrelli, G.: Run-time efficient probabilistic model checking. In: 33rd International Conference on Software Engineering (ICSE 2011), pp. 341–350 (2011)

    Google Scholar 

  36. Filieri, A., Ghezzi, C., Tamburrelli, G.: A formal approach to adaptive software: continuous assurance of non-functional requirements. Formal Aspects Comput. 24(2), 163–186 (2012)

    Article  MathSciNet  MATH  Google Scholar 

  37. Filieri, A., Tamburrelli, G.: Probabilistic verification at runtime for self-adaptive systems. In: Cámara, J., de Lemos, R., Ghezzi, C., Lopes, A. (eds.) Assurances for Self-Adaptive Systems: Principles, Models, and Techniques. LNCS, vol. 7740, pp. 30–59. Springer, Heidelberg (2013). https://doi.org/10.1007/978-3-642-36249-1_2

    Chapter  Google Scholar 

  38. Forejt, V., Kwiatkowska, M., Parker, D., Qu, H., Ujma, M.: Incremental runtime verification of probabilistic systems. Technical report RR-12-05, Department of Computer Science, University of Oxford (2012)

    Google Scholar 

  39. Forejt, V., Kwiatkowska, M., Parker, D., Qu, H., Ujma, M.: Incremental runtime verification of probabilistic systems. In: Qadeer, S., Tasiran, S. (eds.) RV 2012. LNCS, vol. 7687, pp. 314–319. Springer, Heidelberg (2013). https://doi.org/10.1007/978-3-642-35632-2_30

    Chapter  Google Scholar 

  40. Gerasimou, S., Calinescu, R., Banks, A.: Efficient runtime quantitative verification using caching, lookahead, and nearly-optimal reconfiguration. In: 9th International Symposium on Software Engineering for Adaptive and Self-Managing Systems (SEAMS 2014), pp. 115–124 (2014)

    Google Scholar 

  41. Gerasimou, S., Tamburrelli, G., Calinescu, R.: Search-based synthesis of probabilistic models for quality-of-service software engineering. In: 30th IEEE/ACM International Conference on Automated Software Engineering (ASE), pp. 319–330, November 2015

    Google Scholar 

  42. Ghezzi, C.: Evolution, adaptation, and the quest for incrementality. In: Calinescu, R., Garlan, D. (eds.) Monterey Workshop 2012. LNCS, vol. 7539, pp. 369–379. Springer, Heidelberg (2012). https://doi.org/10.1007/978-3-642-34059-8_19

    Chapter  Google Scholar 

  43. Ghezzi, C., Greenyer, J., La Manna, V.P.: Synthesizing dynamically updating controllers from changes in scenario-based specifications. In: Proceedings of the 7th International Symposium on Software Engineering for Adaptive and Self-Managing Systems, SEAMS 2012, pp. 145–154. IEEE Press, Piscataway (2012)

    Google Scholar 

  44. Ghezzi, C., Pezzè, M., Sama, M., Tamburrelli, G.: Mining behavior models from user-intensive web applications. In: 36th International Conference on Software Engineering (ICSE 2014), pp. 277–287 (2014)

    Google Scholar 

  45. Hahn, E., Hermanns, H., Zhang, L.: Probabilistic reachability for parametric Markov models. Int. J. Softw. Tools Technol. Transfer 13(1), 3–19 (2011)

    Article  Google Scholar 

  46. Hahn, E.M., Han, T., Zhang, L.: Synthesis for PCTL in parametric Markov decision processes. In: Bobaru, M., Havelund, K., Holzmann, G.J., Joshi, R. (eds.) NFM 2011. LNCS, vol. 6617, pp. 146–161. Springer, Heidelberg (2011). https://doi.org/10.1007/978-3-642-20398-5_12

    Chapter  Google Scholar 

  47. Hahn, E.M., Hermanns, H., Wachter, B., Zhang, L.: PARAM: a model checker for parametric Markov models. In: Touili, T., Cook, B., Jackson, P. (eds.) CAV 2010. LNCS, vol. 6174, pp. 660–664. Springer, Heidelberg (2010). https://doi.org/10.1007/978-3-642-14295-6_56

    Chapter  Google Scholar 

  48. Hahn, E.M., Hermanns, H., Zhang, L.: Probabilistic reachability for parametric Markov models. In: Păsăreanu, C.S. (ed.) SPIN 2009. LNCS, vol. 5578, pp. 88–106. Springer, Heidelberg (2009). https://doi.org/10.1007/978-3-642-02652-2_10

    Chapter  Google Scholar 

  49. Harman, M., Mansouri, S.A., Zhang, Y.: Search-based software engineering: trends, techniques and applications. ACM Comput. Surv. 45(1), 11:1–11:61 (2012)

    Article  Google Scholar 

  50. Hoare, C.A.R.: An axiomatic basis for computer programming. Commun. ACM 12(10), 576–580 (1969)

    Article  MATH  Google Scholar 

  51. Johnson, K., Calinescu, R., Kikuchi, S.: An incremental verification framework for component-based software systems. In: 16th International Symposium on Component-Based Software Engineering (CBSE 2013), pp. 33–42 (2013)

    Google Scholar 

  52. Katoen, J.-P., Zapreev, I.S., Hahn, E.M., Hermanns, H., Jansen, D.N.: The ins and outs of the probabilistic model checker MRMC. Perform. Eval. 68(2), 90–104 (2011)

    Article  Google Scholar 

  53. Kephart, J., Chess, D.: The vision of autonomic computing. Computer 36(1), 41–50 (2003)

    Article  MathSciNet  Google Scholar 

  54. Komuravelli, A., Pasareanu, C.S., Clarke, E.M.: Learning probabilistic systems from tree samples. In: 27th IEEE/ACM Symposium on Logic in Computer Science (LICS 2012), pp. 441–450 (2012)

    Google Scholar 

  55. Kwiatkowska, M.: Quantitative verification: models, techniques and tools. In: 6th Joint Meeting of the European Software Engineering Conference and the ACM SIGSOFT Symposium on the Foundations of Software Engineering (ESEC/FSE 2007), pp. 449–458 (2007)

    Google Scholar 

  56. Kwiatkowska, M., Norman, G., Parker, D.: PRISM 4.0: verification of probabilistic real-time systems. In: Gopalakrishnan, G., Qadeer, S. (eds.) CAV 2011. LNCS, vol. 6806, pp. 585–591. Springer, Heidelberg (2011). https://doi.org/10.1007/978-3-642-22110-1_47

    Chapter  Google Scholar 

  57. Kwiatkowska, M., Norman, G., Parker, D., Qu, H.: Assume-guarantee verification for probabilistic systems. In: Esparza, J., Majumdar, R. (eds.) TACAS 2010. LNCS, vol. 6015, pp. 23–37. Springer, Heidelberg (2010). https://doi.org/10.1007/978-3-642-12002-2_3

    Chapter  Google Scholar 

  58. Kwiatkowska, M., Norman, G., Parker, D., Sproston, J.: Performance analysis of probabilistic timed automata using digital clocks. Formal Methods Syst. Des. 29(1), 33–78 (2006)

    Article  MATH  Google Scholar 

  59. Kwiatkowska, M., Parker, D., Qu, H.: Incremental quantitative verification for Markov decision processes. In: 41st IEEE/IFIP International Conference on Dependable Systems Networks (DSN 2011), pp. 359–370 (2011)

    Google Scholar 

  60. Meedeniya, I., Grunske, L.: An efficient method for architecture-based reliability evaluation for evolving systems with changing parameters. In: 21st IEEE International Symposium on Software Reliability Engineering (ISSRE 2010), pp. 229–238 (2010)

    Google Scholar 

  61. Nallur, V., Bahsoon, R.: A decentralized self-adaptation mechanism for service-based applications in the cloud. IEEE Trans. Softw. Eng. 39(5), 591–612 (2013)

    Article  Google Scholar 

  62. Naskos, A., Stachtiari, E., Gounaris, A., Katsaros, P., Tsoumakos, D., Konstantinou, I., Sioutas, S.: Dependable horizontal scaling based on probabilistic model checking. In: 15th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (CCGrid 2015) (2015)

    Google Scholar 

  63. Pnueli, A.: In transition from global to modular temporal reasoning about programs. In: Apt, K.R. (ed.) Logics and Models of Concurrent Systems, pp. 123–144. Springer, New York (1985). https://doi.org/10.1007/978-3-642-82453-1_5

    Chapter  Google Scholar 

  64. Segala, R., Lynch, N.A.: Probabilistic simulations for probabilistic processes. Nord. J. Comput. 2(2), 250–273 (1995)

    MathSciNet  MATH  Google Scholar 

  65. Sen, K., Viswanathan, M., Agha, G.: Learning continuous time Markov chains from sample executions. In: Quantitative Evaluation of Systems, pp. 146–155 (2004)

    Google Scholar 

  66. Sykes, D., Magee, J., Kramer, J.: Flashmob: distributed adaptive self-assembly. In: 6th International Symposium on Software Engineering for Adaptive and Self-Managing Systems (SEAMS 2011), pp. 100–109 (2011)

    Google Scholar 

  67. Weyns, D., Calinescu, R.: Tele assistance system: an examplar for self-adaptive service-based systems. In: 10th International Symposium on Software Engineering for Adaptive and Self-Managing Systems (SEAMS 2015) (2015, to appear)

    Google Scholar 

  68. Weyns, D., Haesevoets, R., Helleboogh, A., Holvoet, T., Joosen, W.: The MACODO middleware for context-driven dynamic agent organizations. ACM Trans. Auton. Adapt. Syst. 5(1), 3:1–3:28 (2010)

    Article  Google Scholar 

  69. Weyns, D., Malek, S., Andersson, J.: On decentralized self-adaptation: lessons from the trenches and challenges for the future. In: 5th International Symposium on Software Engineering for Adaptive and Self-Managing Systems (SEAMS 2010), pp. 84–93 (2010)

    Google Scholar 

  70. Weyns, D., et al.: On patterns for decentralized control in self-adaptive systems. In: de Lemos, R., Giese, H., Müller, H.A., Shaw, M. (eds.) Software Engineering for Self-Adaptive Systems II. LNCS, vol. 7475, pp. 76–107. Springer, Heidelberg (2013). https://doi.org/10.1007/978-3-642-35813-5_4

    Chapter  Google Scholar 

  71. Wongpiromsarn, T., Ulusoy, A., Belta, C., Frazzoli, E., Rus, D.: Incremental synthesis of control policies for heterogeneous multi-agent systems with linear temporal logic specifications. In: 2013 IEEE International Conference on Robotics and Automation (ICRA), pp. 5011–5018, May 2013

    Google Scholar 

  72. Woodcock, J., Larsen, P.G., Bicarregui, J., Fitzgerald, J.: Formal methods: practice and experience. ACM Comput. Surv. 41(4), 19:1–19:36 (2009)

    Article  Google Scholar 

  73. Younes, H.L.S.: Ymer: a statistical model checker. In: Etessami, K., Rajamani, S.K. (eds.) CAV 2005. LNCS, vol. 3576, pp. 429–433. Springer, Heidelberg (2005). https://doi.org/10.1007/11513988_43

    Chapter  Google Scholar 

  74. Zeng, L., Benatallah, B., Ngu, A., Dumas, M., Kalagnanam, J., Chang, H.: QoS-aware middleware for web services composition. IEEE Trans. Softw. Eng. 30(5), 311–327 (2004)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science, University of York, York, UK

    Radu Calinescu, Simos Gerasimou & Colin Paterson

  2. School of Computer and Mathematical Science, Auckland University of Technology, Auckland, New Zealand

    Kenneth Johnson

Authors
  1. Radu Calinescu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Simos Gerasimou
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Kenneth Johnson
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Colin Paterson
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Radu Calinescu .

Editor information

Editors and Affiliations

  1. University of Kent, Canterbury, United Kingdom

    Rogério de Lemos

  2. Carnegie Mellon University, Pittsburgh, Pennsylvania, USA

    David Garlan

  3. Politecnico di Milano, Milan, Italy

    Carlo Ghezzi

  4. Hasso Plattner Institute for Software Systems Engineering, Potsdam, Germany

    Holger Giese

Rights and permissions

Reprints and permissions

Copyright information

© 2017 Springer International Publishing AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Calinescu, R., Gerasimou, S., Johnson, K., Paterson, C. (2017). Using Runtime Quantitative Verification to Provide Assurance Evidence for Self-Adaptive Software. In: de Lemos, R., Garlan, D., Ghezzi, C., Giese, H. (eds) Software Engineering for Self-Adaptive Systems III. Assurances. Lecture Notes in Computer Science(), vol 9640. Springer, Cham. https://doi.org/10.1007/978-3-319-74183-3_8

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-74183-3_8

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-74182-6

  • Online ISBN: 978-3-319-74183-3

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation