Abstract
Twenty years have passed since the first workshop on runtime verification—the area has grown and evolved with hundreds of papers published and a sizeable number of mature tools developed. In a special occasion like this it is good to look back, but it is also good to look forward to the future. In this paper, we outline a very brief history of runtime verification, and propose a way of passing the knowledge down to future generations of academics and industry practitioners in the form of a roadmap for teaching runtime verification. The proposal, based on our experience, not only equips students with the fundamental theory underpinning runtime verification, but also ensures they have the required skills to engineer it into industrial systems. Our hope is that this would increase uptake and eventually give rise to the establishment of industry-grade tools.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
- 1.
- 2.
The workshop was held for many years till it became a conference in 2010 [BFF+10]; the conference has held every year since then.
- 3.
- 4.
- 5.
One may prefer the term actuators instead in order to not confuse it with the controllers as used in the controller synthesis research area (they have similar purpose but we are not here concerned with synthesising controllers that are correct for all executions of the SUS).
- 6.
There are exceptions though. For instance, if the observer monitor just logs all events in the execution trace, there is no need to write any formal specification.
- 7.
Reference [50] in the quote appears as [HW08] here.
- 8.
- 9.
Point as in being possible to verify by checking for a particular condition at a particular point in the control logic of the system.
- 10.
In a course using Java one can adopt a tool such as AspectJ.
- 11.
We use standard regular expression syntax here, with a indicating the occurrence of that event, \(\overline{a}\) to indicate the occurrence of any event other than a, \(e\cdot e'\) to indicate sequential composition of two sub-expressions, \(e+e'\) to indicate the choice and \(e^{*}\) to indicate any number of repetitions.
- 12.
It is worth highlighting that the authors initial work in runtime verification was done jointly. However, some of the discussion which follows, regarding ongoing collaborative projects and lecturing is specific to the University of Malta, although much of what we describe was discussed between all three authors at the time.
- 13.
The transaction object before serialisation and the object after serialisation should be treated as the same object based on a logical identifier.
- 14.
There were other advantages for these choices, such as the possibility of also manipulating the monitored system state (as a reaction to observations), and the seamless introduction of a stopwatch API. However, these aspects fall outside the scope of this paper.
- 15.
Anyone interested in using the material may get in touch with the authors.
- 16.
- 17.
- 18.
References
Agrawal, S., Bonakdarpour, B.: Runtime verification of \(k\)-safety hyperproperties in HyperLTL. In: CSF 2016, pp. 239–252. IEEE CS Press (2016)
Azzopardi, S., Colombo, C., Pace, G.J.: Control-flow residual analysis for symbolic automata. In: PrePost@iFM 2017, Volume 254 of EPTCS, pp. 29–43 (2017)
Ahrendt, W., Chimento, J.M., Pace, G.J., Schneider, G.: Verifying data- and control-oriented properties combining static and runtime verification: theory and tools. Formal Methods Syst. Des. 51(1), 200–265 (2017). https://doi.org/10.1007/s10703-017-0274-y
Bonakdarpour, B., Finkbeiner, B.: The complexity of monitoring hyperproperties. In: CSF 2018, pp. 162–174. IEEE CS Press (2018)
Bartocci, E., et al.: First international competition on runtime verification: rules, benchmarks, tools, and final results of CRV 2014. Int. J. Softw. Tools Technol. Transf. 21(1), 31–70 (2019). https://doi.org/10.1007/s10009-017-0454-5
Barringer, H., et al. (eds.): RV 2010. LNCS, vol. 6418. Springer, Heidelberg (2010). https://doi.org/10.1007/978-3-642-16612-9
Barringer, H., Falcone, Y., Havelund, K., Reger, G., Rydeheard, D.: Quantified event automata: towards expressive and efficient runtime monitors. In: Giannakopoulou, D., Méry, D. (eds.) FM 2012. LNCS, vol. 7436, pp. 68–84. Springer, Heidelberg (2012). https://doi.org/10.1007/978-3-642-32759-9_9
Barringer, H., Goldberg, A., Havelund, K., Sen, K.: Rule-based runtime verification. In: Steffen, B., Levi, G. (eds.) VMCAI 2004. LNCS, vol. 2937, pp. 44–57. Springer, Heidelberg (2004). https://doi.org/10.1007/978-3-540-24622-0_5
Barringer, H., Groce, A., Havelund, K., Smith, M.H.: An entry point for formal methods: specification and analysis of event logs. In: FMA 2009, Volume 20 of EPTCS, pp. 16–21 (2009)
Bodden, E., Havelund, K.: Aspect-oriented race detection in Java. IEEE Trans. Softw. Eng. 36(4), 509–527 (2010)
Bodden, E., Lam, P.: Clara: partially evaluating runtime monitors at compile time. In: Barringer, H., et al. (eds.) RV 2010. LNCS, vol. 6418, pp. 74–88. Springer, Heidelberg (2010). https://doi.org/10.1007/978-3-642-16612-9_8
Bauer, A., Leucker, M., Schallhart, C.: The good, the bad, and the ugly, but how ugly is ugly? In: Sokolsky, O., Taşıran, S. (eds.) RV 2007. LNCS, vol. 4839, pp. 126–138. Springer, Heidelberg (2007). https://doi.org/10.1007/978-3-540-77395-5_11
Bauer, A., Leucker, M., Schallhart, C.: Runtime verification for LTL and TLTL. ACM Trans. Softw. Eng. Methodol. 20(4), 14 (2011)
Brzozowski, J.A.: Derivatives of regular expressions. J. ACM 11(4), 481–494 (1964)
Brett, N., Siddique, U., Bonakdarpour, B.: Rewriting-based runtime verification for alternation-free HyperLTL. In: Legay, A., Margaria, T. (eds.) TACAS 2017. LNCS, vol. 10206, pp. 77–93. Springer, Heidelberg (2017). https://doi.org/10.1007/978-3-662-54580-5_5
Chimento, J.M., Ahrendt, W., Pace, G.J., Schneider, G.: StaRVOOrS : a tool for combined static and runtime verification of Java. In: Bartocci, E., Majumdar, R. (eds.) RV 2015. LNCS, vol. 9333, pp. 297–305. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-23820-3_21
Chimento, J.M., Ahrendt, W., Schneider, G.: Testing meets static and runtime verification. In: FormaliSE 2018, pp. 30–39. ACM (2018)
Cauchi, A., Colombo, C., Francalanza, A., Micallef, M., Pace, G.J.: Using gherkin to extract tests and monitors for safer medical device interaction design. In: ACM SIGCHI EICS 2016, pp. 275–280. ACM (2016)
Clarkson, M.R., Finkbeiner, B., Koleini, M., Micinski, K.K., Rabe, M.N., Sánchez, C.: Temporal logics for hyperproperties. In: Abadi, M., Kremer, S. (eds.) POST 2014. LNCS, vol. 8414, pp. 265–284. Springer, Heidelberg (2014). https://doi.org/10.1007/978-3-642-54792-8_15
Colombo, C., Pace, G.J.: Considering academia-industry projects meta-characteristics in runtime verification design. In: Margaria, T., Steffen, B. (eds.) ISoLA 2018. LNCS, vol. 11247, pp. 32–41. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-03427-6_5
Colombo, C., Pace, G.J.: Industrial experiences with runtime verification of financial transaction systems: lessons learnt and standing challenges. In: Bartocci, E., Falcone, Y. (eds.) Lectures on Runtime Verification. LNCS, vol. 10457, pp. 211–232. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-75632-5_7
Colombo, C., Pace, G.J., Abela, P.: Safer asynchronous runtime monitoring using compensations. Formal Methods Syst. Des. 41(3), 269–294 (2012). https://doi.org/10.1007/s10703-012-0142-8
Colombo, C., Pace, G.J., Schneider, G.: Dynamic event-based runtime monitoring of real-time and contextual properties. In: Cofer, D., Fantechi, A. (eds.) FMICS 2008. LNCS, vol. 5596, pp. 135–149. Springer, Heidelberg (2009). https://doi.org/10.1007/978-3-642-03240-0_13
Colombo, C., Pace, G.J., Schneider, G.: LARVA – safer monitoring of real-time Java programs (tool paper). In: SEFM 2009, pp. 33–37. IEEE Computer Society (2009)
Colombo, C., Pace, G.J., Schneider, G.: Safe runtime verification of real-time properties. In: Ouaknine, J., Vaandrager, F.W. (eds.) FORMATS 2009. LNCS, vol. 5813, pp. 103–117. Springer, Heidelberg (2009). https://doi.org/10.1007/978-3-642-04368-0_10
Chen, F., Rosu, G.: Towards monitoring-oriented programming: a paradigm combining specification and implementation. In: RV 2003, vol. 89, pp. 108–127. ENTCS (2003)
Chen, F., Serbanuta, T.-F., Rosu, G.: jPredictor: a predictive runtime analysis tool for Gava. In: ICSE 2008, pp. 221–230. ACM (2008)
Decker, N., Leucker, M., Thoma, D.: jUnitRV–adding runtime verification to jUnit. In: Brat, G., Rungta, N., Venet, A. (eds.) NFM 2013. LNCS, vol. 7871, pp. 459–464. Springer, Heidelberg (2013). https://doi.org/10.1007/978-3-642-38088-4_34
Drusinsky, D.: The temporal rover and the ATG rover. In: Havelund, K., Penix, J., Visser, W. (eds.) SPIN 2000. LNCS, vol. 1885, pp. 323–330. Springer, Heidelberg (2000). https://doi.org/10.1007/10722468_19
Falcone, Y., Fernandez, J.-C., Mounier, L.: Runtime verification of safety-progress properties. In: Bensalem, S., Peled, D.A. (eds.) RV 2009. LNCS, vol. 5779, pp. 40–59. Springer, Heidelberg (2009). https://doi.org/10.1007/978-3-642-04694-0_4
Falcone, Y., Fernandez, J.-C., Mounier, L.: What can you verify and enforce at runtime? Int. J. Softw. Tools Technol. Transf. (STTT) 14(3), 349–382 (2012). https://doi.org/10.1007/s10009-011-0196-8
Finkbeiner, B., Hahn, C., Stenger, M., Tentrup, L.: Monitoring hyperproperties. In: Lahiri, S., Reger, G. (eds.) RV 2017. LNCS, vol. 10548, pp. 190–207. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-67531-2_12
Falcone, Y., Krstić, S., Reger, G., Traytel, D.: A taxonomy for classifying runtime verification tools. In: Colombo, C., Leucker, M. (eds.) RV 2018. LNCS, vol. 11237, pp. 241–262. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-03769-7_14
Giannakopoulou, D., Havelund, K.: Automata-based verification of temporal properties on running programs. In: ASE 2001, pp. 412–416. IEEE Computer Society (2001)
Harrow, J.J.: Runtime checking of multithreaded applications with visual threads. In: Havelund, K., Penix, J., Visser, W. (eds.) SPIN 2000. LNCS, vol. 1885, pp. 331–342. Springer, Heidelberg (2000). https://doi.org/10.1007/10722468_20
Havelund, K.: Rule-based runtime verification revisited. Int. J. Softw. Tools Technol. Transf. 17(2), 143–170 (2015). https://doi.org/10.1007/s10009-014-0309-2
Havelund, K., Peled, D.: Runtime verification: from propositional to first-order temporal logic. In: Colombo, C., Leucker, M. (eds.) RV 2018. LNCS, vol. 11237, pp. 90–112. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-03769-7_7
Havelund, K., Rosu, G.: Monitoring Java programs with Java PathExplorer. ENTCS 55(2), 200–217 (2001)
Havelund, K., Rosu, G.: Monitoring programs using rewriting. In: ASE 2001, pp. 135–143 (2001)
Havelund, K., Roşu, G.: Synthesizing monitors for safety properties. In: Katoen, J.-P., Stevens, P. (eds.) TACAS 2002. LNCS, vol. 2280, pp. 342–356. Springer, Heidelberg (2002). https://doi.org/10.1007/3-540-46002-0_24
Havelund, K., Roşu, G.: Runtime verification - 17 years later. In: Colombo, C., Leucker, M. (eds.) RV 2018. LNCS, vol. 11237, pp. 3–17. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-03769-7_1
Havelund, K., Reger, G., Roşu, G.: Runtime verification past experiences and future projections. In: Steffen, B., Woeginger, G. (eds.) Computing and Software Science. LNCS, vol. 10000, pp. 532–562. Springer, Cham (2019). https://doi.org/10.1007/978-3-319-91908-9_25
Havelund, K., Van Wyk, E.: Aspect-oriented monitoring of C programs. In: IARP-IEEE/RAS-EURON 2008 (2008)
Kim, M., Kannan, S., Lee, I., Sokolsky, O., Viswanathan, M.: Computational analysis of run-time monitoring - fundamentals of Java-MaC. In: RV 2002, Volume 70 of ENTCS, pp. 80–94 (2002)
Leucker, M.: Teaching runtime verification. In: Khurshid, S., Sen, K. (eds.) RV 2011. LNCS, vol. 7186, pp. 34–48. Springer, Heidelberg (2012). https://doi.org/10.1007/978-3-642-29860-8_4
Pinisetty, S., Preoteasa, V., Tripakis, S., Jéron, T., Falcone, Y., Marchand, H.: Predictive runtime enforcement. In: ACM SAC 2016, pp. 1628–1633. ACM (2016)
Pnueli, A., Zaks, A.: PSL model checking and run-time verification via testers. In: Misra, J., Nipkow, T., Sekerinski, E. (eds.) FM 2006. LNCS, vol. 4085, pp. 573–586. Springer, Heidelberg (2006). https://doi.org/10.1007/11813040_38
Sammapun, U., Easwaran, A., Lee, I., Sokolsky, O.: Simulation of simultaneous events in regular expressions for run-time verification. In: RV 2004, Volume 113 of ENTCS, pp. 123–143 (2005)
Sánchez, C., et al.: A survey of challenges for runtime verification from advanced application domains (beyond software). Formal Methods Syst. Des. 54(3), 279–335 (2019). https://doi.org/10.1007/s10703-019-00337-w
Stucki, S., Sánchez, C., Schneider, G., Bonakdarpour, B.: Gray-box monitoring of hyperproperties. In: ter Beek, M.H., McIver, A., Oliveira, J.N. (eds.) FM 2019. LNCS, vol. 11800, pp. 406–424. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-30942-8_25
Zhang, X., Leucker, M., Dong, W.: Runtime verification with predictive semantics. In: Goodloe, A.E., Person, S. (eds.) NFM 2012. LNCS, vol. 7226, pp. 418–432. Springer, Heidelberg (2012). https://doi.org/10.1007/978-3-642-28891-3_37
Acknowledgements
This research has been partially supported by the Swedish Research Council (Vetenskapsrådet) under Grant 2015-04154 “PolUser”.
Author information
Authors and Affiliations
Corresponding authors
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this chapter
Cite this chapter
Colombo, C., Pace, G.J., Schneider, G. (2021). Runtime Verification: Passing on the Baton. In: Bartocci, E., Falcone, Y., Leucker, M. (eds) Formal Methods in Outer Space. Lecture Notes in Computer Science(), vol 13065. Springer, Cham. https://doi.org/10.1007/978-3-030-87348-6_5
Download citation
DOI: https://doi.org/10.1007/978-3-030-87348-6_5
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-87347-9
Online ISBN: 978-3-030-87348-6
eBook Packages: Computer ScienceComputer Science (R0)