Skip to main content
SpringerLink
Log in

Runtime Verification: Passing on the Baton

  • Chapter
  • First Online:
Formal Methods in Outer Space

Part of the book series: Lecture Notes in Computer Science ((LNPSE,volume 13065))

  • 365 Accesses

Abstract

Twenty years have passed since the first workshop on runtime verification—the area has grown and evolved with hundreds of papers published and a sizeable number of mature tools developed. In a special occasion like this it is good to look back, but it is also good to look forward to the future. In this paper, we outline a very brief history of runtime verification, and propose a way of passing the knowledge down to future generations of academics and industry practitioners in the form of a roadmap for teaching runtime verification. The proposal, based on our experience, not only equips students with the fundamental theory underpinning runtime verification, but also ensures they have the required skills to engineer it into industrial systems. Our hope is that this would increase uptake and eventually give rise to the establishment of industry-grade tools.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 44.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 59.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    http://www.runtime-verification.org/rv2001.

  2. 2.

    The workshop was held for many years till it became a conference in 2010 [BFF+10]; the conference has held every year since then.

  3. 3.

    https://www.havelund.com/.

  4. 4.

    In this paper, reference [30] appears as [Dru00], and reference [36] as [Har00].

  5. 5.

    One may prefer the term actuators instead in order to not confuse it with the controllers as used in the controller synthesis research area (they have similar purpose but we are not here concerned with synthesising controllers that are correct for all executions of the SUS).

  6. 6.

    There are exceptions though. For instance, if the observer monitor just logs all events in the execution trace, there is no need to write any formal specification.

  7. 7.

    Reference [50] in the quote appears as [HW08] here.

  8. 8.

    https://www.cost.eu/actions/IC1402.

  9. 9.

    Point as in being possible to verify by checking for a particular condition at a particular point in the control logic of the system.

  10. 10.

    In a course using Java one can adopt a tool such as AspectJ.

  11. 11.

    We use standard regular expression syntax here, with a indicating the occurrence of that event, \(\overline{a}\) to indicate the occurrence of any event other than a, \(e\cdot e'\) to indicate sequential composition of two sub-expressions, \(e+e'\) to indicate the choice and \(e^{*}\) to indicate any number of repetitions.

  12. 12.

    It is worth highlighting that the authors initial work in runtime verification was done jointly. However, some of the discussion which follows, regarding ongoing collaborative projects and lecturing is specific to the University of Malta, although much of what we describe was discussed between all three authors at the time.

  13. 13.

    The transaction object before serialisation and the object after serialisation should be treated as the same object based on a logical identifier.

  14. 14.

    There were other advantages for these choices, such as the possibility of also manipulating the monitored system state (as a reaction to observations), and the seamless introduction of a stopwatch API. However, these aspects fall outside the scope of this paper.

  15. 15.

    Anyone interested in using the material may get in touch with the authors.

  16. 16.

    https://meals-project.eu/node/67.

  17. 17.

    https://rv2016.imag.fr/?page_id=128.

  18. 18.

    https://www.youtube.com/watch?v=Vyz6kte4PVk.

References

  1. Agrawal, S., Bonakdarpour, B.: Runtime verification of \(k\)-safety hyperproperties in HyperLTL. In: CSF 2016, pp. 239–252. IEEE CS Press (2016)

    Google Scholar 

  2. Azzopardi, S., Colombo, C., Pace, G.J.: Control-flow residual analysis for symbolic automata. In: PrePost@iFM 2017, Volume 254 of EPTCS, pp. 29–43 (2017)

    Google Scholar 

  3. Ahrendt, W., Chimento, J.M., Pace, G.J., Schneider, G.: Verifying data- and control-oriented properties combining static and runtime verification: theory and tools. Formal Methods Syst. Des. 51(1), 200–265 (2017). https://doi.org/10.1007/s10703-017-0274-y

    Article  MATH  Google Scholar 

  4. Bonakdarpour, B., Finkbeiner, B.: The complexity of monitoring hyperproperties. In: CSF 2018, pp. 162–174. IEEE CS Press (2018)

    Google Scholar 

  5. Bartocci, E., et al.: First international competition on runtime verification: rules, benchmarks, tools, and final results of CRV 2014. Int. J. Softw. Tools Technol. Transf. 21(1), 31–70 (2019). https://doi.org/10.1007/s10009-017-0454-5

    Article  Google Scholar 

  6. Barringer, H., et al. (eds.): RV 2010. LNCS, vol. 6418. Springer, Heidelberg (2010). https://doi.org/10.1007/978-3-642-16612-9

    Book  MATH  Google Scholar 

  7. Barringer, H., Falcone, Y., Havelund, K., Reger, G., Rydeheard, D.: Quantified event automata: towards expressive and efficient runtime monitors. In: Giannakopoulou, D., Méry, D. (eds.) FM 2012. LNCS, vol. 7436, pp. 68–84. Springer, Heidelberg (2012). https://doi.org/10.1007/978-3-642-32759-9_9

    Chapter  Google Scholar 

  8. Barringer, H., Goldberg, A., Havelund, K., Sen, K.: Rule-based runtime verification. In: Steffen, B., Levi, G. (eds.) VMCAI 2004. LNCS, vol. 2937, pp. 44–57. Springer, Heidelberg (2004). https://doi.org/10.1007/978-3-540-24622-0_5

    Chapter  MATH  Google Scholar 

  9. Barringer, H., Groce, A., Havelund, K., Smith, M.H.: An entry point for formal methods: specification and analysis of event logs. In: FMA 2009, Volume 20 of EPTCS, pp. 16–21 (2009)

    Google Scholar 

  10. Bodden, E., Havelund, K.: Aspect-oriented race detection in Java. IEEE Trans. Softw. Eng. 36(4), 509–527 (2010)

    Article  Google Scholar 

  11. Bodden, E., Lam, P.: Clara: partially evaluating runtime monitors at compile time. In: Barringer, H., et al. (eds.) RV 2010. LNCS, vol. 6418, pp. 74–88. Springer, Heidelberg (2010). https://doi.org/10.1007/978-3-642-16612-9_8

    Chapter  MATH  Google Scholar 

  12. Bauer, A., Leucker, M., Schallhart, C.: The good, the bad, and the ugly, but how ugly is ugly? In: Sokolsky, O., Taşıran, S. (eds.) RV 2007. LNCS, vol. 4839, pp. 126–138. Springer, Heidelberg (2007). https://doi.org/10.1007/978-3-540-77395-5_11

    Chapter  MATH  Google Scholar 

  13. Bauer, A., Leucker, M., Schallhart, C.: Runtime verification for LTL and TLTL. ACM Trans. Softw. Eng. Methodol. 20(4), 14 (2011)

    Article  Google Scholar 

  14. Brzozowski, J.A.: Derivatives of regular expressions. J. ACM 11(4), 481–494 (1964)

    Article  MathSciNet  Google Scholar 

  15. Brett, N., Siddique, U., Bonakdarpour, B.: Rewriting-based runtime verification for alternation-free HyperLTL. In: Legay, A., Margaria, T. (eds.) TACAS 2017. LNCS, vol. 10206, pp. 77–93. Springer, Heidelberg (2017). https://doi.org/10.1007/978-3-662-54580-5_5

    Chapter  MATH  Google Scholar 

  16. Chimento, J.M., Ahrendt, W., Pace, G.J., Schneider, G.: StaRVOOrS : a tool for combined static and runtime verification of Java. In: Bartocci, E., Majumdar, R. (eds.) RV 2015. LNCS, vol. 9333, pp. 297–305. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-23820-3_21

    Chapter  MATH  Google Scholar 

  17. Chimento, J.M., Ahrendt, W., Schneider, G.: Testing meets static and runtime verification. In: FormaliSE 2018, pp. 30–39. ACM (2018)

    Google Scholar 

  18. Cauchi, A., Colombo, C., Francalanza, A., Micallef, M., Pace, G.J.: Using gherkin to extract tests and monitors for safer medical device interaction design. In: ACM SIGCHI EICS 2016, pp. 275–280. ACM (2016)

    Google Scholar 

  19. Clarkson, M.R., Finkbeiner, B., Koleini, M., Micinski, K.K., Rabe, M.N., Sánchez, C.: Temporal logics for hyperproperties. In: Abadi, M., Kremer, S. (eds.) POST 2014. LNCS, vol. 8414, pp. 265–284. Springer, Heidelberg (2014). https://doi.org/10.1007/978-3-642-54792-8_15

    Chapter  Google Scholar 

  20. Colombo, C., Pace, G.J.: Considering academia-industry projects meta-characteristics in runtime verification design. In: Margaria, T., Steffen, B. (eds.) ISoLA 2018. LNCS, vol. 11247, pp. 32–41. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-03427-6_5

    Chapter  Google Scholar 

  21. Colombo, C., Pace, G.J.: Industrial experiences with runtime verification of financial transaction systems: lessons learnt and standing challenges. In: Bartocci, E., Falcone, Y. (eds.) Lectures on Runtime Verification. LNCS, vol. 10457, pp. 211–232. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-75632-5_7

    Chapter  Google Scholar 

  22. Colombo, C., Pace, G.J., Abela, P.: Safer asynchronous runtime monitoring using compensations. Formal Methods Syst. Des. 41(3), 269–294 (2012). https://doi.org/10.1007/s10703-012-0142-8

    Article  MATH  Google Scholar 

  23. Colombo, C., Pace, G.J., Schneider, G.: Dynamic event-based runtime monitoring of real-time and contextual properties. In: Cofer, D., Fantechi, A. (eds.) FMICS 2008. LNCS, vol. 5596, pp. 135–149. Springer, Heidelberg (2009). https://doi.org/10.1007/978-3-642-03240-0_13

    Chapter  Google Scholar 

  24. Colombo, C., Pace, G.J., Schneider, G.: LARVA – safer monitoring of real-time Java programs (tool paper). In: SEFM 2009, pp. 33–37. IEEE Computer Society (2009)

    Google Scholar 

  25. Colombo, C., Pace, G.J., Schneider, G.: Safe runtime verification of real-time properties. In: Ouaknine, J., Vaandrager, F.W. (eds.) FORMATS 2009. LNCS, vol. 5813, pp. 103–117. Springer, Heidelberg (2009). https://doi.org/10.1007/978-3-642-04368-0_10

    Chapter  MATH  Google Scholar 

  26. Chen, F., Rosu, G.: Towards monitoring-oriented programming: a paradigm combining specification and implementation. In: RV 2003, vol. 89, pp. 108–127. ENTCS (2003)

    Google Scholar 

  27. Chen, F., Serbanuta, T.-F., Rosu, G.: jPredictor: a predictive runtime analysis tool for Gava. In: ICSE 2008, pp. 221–230. ACM (2008)

    Google Scholar 

  28. Decker, N., Leucker, M., Thoma, D.: jUnitRV–adding runtime verification to jUnit. In: Brat, G., Rungta, N., Venet, A. (eds.) NFM 2013. LNCS, vol. 7871, pp. 459–464. Springer, Heidelberg (2013). https://doi.org/10.1007/978-3-642-38088-4_34

    Chapter  Google Scholar 

  29. Drusinsky, D.: The temporal rover and the ATG rover. In: Havelund, K., Penix, J., Visser, W. (eds.) SPIN 2000. LNCS, vol. 1885, pp. 323–330. Springer, Heidelberg (2000). https://doi.org/10.1007/10722468_19

    Chapter  MATH  Google Scholar 

  30. Falcone, Y., Fernandez, J.-C., Mounier, L.: Runtime verification of safety-progress properties. In: Bensalem, S., Peled, D.A. (eds.) RV 2009. LNCS, vol. 5779, pp. 40–59. Springer, Heidelberg (2009). https://doi.org/10.1007/978-3-642-04694-0_4

    Chapter  Google Scholar 

  31. Falcone, Y., Fernandez, J.-C., Mounier, L.: What can you verify and enforce at runtime? Int. J. Softw. Tools Technol. Transf. (STTT) 14(3), 349–382 (2012). https://doi.org/10.1007/s10009-011-0196-8

    Article  Google Scholar 

  32. Finkbeiner, B., Hahn, C., Stenger, M., Tentrup, L.: Monitoring hyperproperties. In: Lahiri, S., Reger, G. (eds.) RV 2017. LNCS, vol. 10548, pp. 190–207. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-67531-2_12

    Chapter  Google Scholar 

  33. Falcone, Y., Krstić, S., Reger, G., Traytel, D.: A taxonomy for classifying runtime verification tools. In: Colombo, C., Leucker, M. (eds.) RV 2018. LNCS, vol. 11237, pp. 241–262. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-03769-7_14

    Chapter  Google Scholar 

  34. Giannakopoulou, D., Havelund, K.: Automata-based verification of temporal properties on running programs. In: ASE 2001, pp. 412–416. IEEE Computer Society (2001)

    Google Scholar 

  35. Harrow, J.J.: Runtime checking of multithreaded applications with visual threads. In: Havelund, K., Penix, J., Visser, W. (eds.) SPIN 2000. LNCS, vol. 1885, pp. 331–342. Springer, Heidelberg (2000). https://doi.org/10.1007/10722468_20

    Chapter  MATH  Google Scholar 

  36. Havelund, K.: Rule-based runtime verification revisited. Int. J. Softw. Tools Technol. Transf. 17(2), 143–170 (2015). https://doi.org/10.1007/s10009-014-0309-2

    Article  Google Scholar 

  37. Havelund, K., Peled, D.: Runtime verification: from propositional to first-order temporal logic. In: Colombo, C., Leucker, M. (eds.) RV 2018. LNCS, vol. 11237, pp. 90–112. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-03769-7_7

    Chapter  Google Scholar 

  38. Havelund, K., Rosu, G.: Monitoring Java programs with Java PathExplorer. ENTCS 55(2), 200–217 (2001)

    Google Scholar 

  39. Havelund, K., Rosu, G.: Monitoring programs using rewriting. In: ASE 2001, pp. 135–143 (2001)

    Google Scholar 

  40. Havelund, K., Roşu, G.: Synthesizing monitors for safety properties. In: Katoen, J.-P., Stevens, P. (eds.) TACAS 2002. LNCS, vol. 2280, pp. 342–356. Springer, Heidelberg (2002). https://doi.org/10.1007/3-540-46002-0_24

    Chapter  MATH  Google Scholar 

  41. Havelund, K., Roşu, G.: Runtime verification - 17 years later. In: Colombo, C., Leucker, M. (eds.) RV 2018. LNCS, vol. 11237, pp. 3–17. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-03769-7_1

    Chapter  MATH  Google Scholar 

  42. Havelund, K., Reger, G., Roşu, G.: Runtime verification past experiences and future projections. In: Steffen, B., Woeginger, G. (eds.) Computing and Software Science. LNCS, vol. 10000, pp. 532–562. Springer, Cham (2019). https://doi.org/10.1007/978-3-319-91908-9_25

    Chapter  Google Scholar 

  43. Havelund, K., Van Wyk, E.: Aspect-oriented monitoring of C programs. In: IARP-IEEE/RAS-EURON 2008 (2008)

    Google Scholar 

  44. Kim, M., Kannan, S., Lee, I., Sokolsky, O., Viswanathan, M.: Computational analysis of run-time monitoring - fundamentals of Java-MaC. In: RV 2002, Volume 70 of ENTCS, pp. 80–94 (2002)

    Google Scholar 

  45. Leucker, M.: Teaching runtime verification. In: Khurshid, S., Sen, K. (eds.) RV 2011. LNCS, vol. 7186, pp. 34–48. Springer, Heidelberg (2012). https://doi.org/10.1007/978-3-642-29860-8_4

    Chapter  Google Scholar 

  46. Pinisetty, S., Preoteasa, V., Tripakis, S., Jéron, T., Falcone, Y., Marchand, H.: Predictive runtime enforcement. In: ACM SAC 2016, pp. 1628–1633. ACM (2016)

    Google Scholar 

  47. Pnueli, A., Zaks, A.: PSL model checking and run-time verification via testers. In: Misra, J., Nipkow, T., Sekerinski, E. (eds.) FM 2006. LNCS, vol. 4085, pp. 573–586. Springer, Heidelberg (2006). https://doi.org/10.1007/11813040_38

    Chapter  Google Scholar 

  48. Sammapun, U., Easwaran, A., Lee, I., Sokolsky, O.: Simulation of simultaneous events in regular expressions for run-time verification. In: RV 2004, Volume 113 of ENTCS, pp. 123–143 (2005)

    Google Scholar 

  49. Sánchez, C., et al.: A survey of challenges for runtime verification from advanced application domains (beyond software). Formal Methods Syst. Des. 54(3), 279–335 (2019). https://doi.org/10.1007/s10703-019-00337-w

    Article  MATH  Google Scholar 

  50. Stucki, S., Sánchez, C., Schneider, G., Bonakdarpour, B.: Gray-box monitoring of hyperproperties. In: ter Beek, M.H., McIver, A., Oliveira, J.N. (eds.) FM 2019. LNCS, vol. 11800, pp. 406–424. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-30942-8_25

    Chapter  Google Scholar 

  51. Zhang, X., Leucker, M., Dong, W.: Runtime verification with predictive semantics. In: Goodloe, A.E., Person, S. (eds.) NFM 2012. LNCS, vol. 7226, pp. 418–432. Springer, Heidelberg (2012). https://doi.org/10.1007/978-3-642-28891-3_37

    Chapter  Google Scholar 

Download references

Acknowledgements

This research has been partially supported by the Swedish Research Council (Vetenskapsrådet) under Grant 2015-04154 “PolUser”.

Author information

Authors and Affiliations

  1. University of Malta, Msida, Malta

    Christian Colombo & Gordon J. Pace

  2. University of Gothenburg, Gothenburg, Sweden

    Gerardo Schneider

Authors
  1. Christian Colombo
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Gordon J. Pace
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Gerardo Schneider
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Christian Colombo , Gordon J. Pace or Gerardo Schneider .

Editor information

Editors and Affiliations

  1. TU Wien, Vienna, Austria

    Ezio Bartocci

  2. Univ. Grenoble Alpes, Inria, CNRS, Grenoble INP, LIG, Grenoble, France

    Yliès Falcone

  3. University of LĂĽbeck, LĂĽbeck, Germany

    Martin Leucker

Rights and permissions

Reprints and permissions

Copyright information

© 2021 Springer Nature Switzerland AG

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Colombo, C., Pace, G.J., Schneider, G. (2021). Runtime Verification: Passing on the Baton. In: Bartocci, E., Falcone, Y., Leucker, M. (eds) Formal Methods in Outer Space. Lecture Notes in Computer Science(), vol 13065. Springer, Cham. https://doi.org/10.1007/978-3-030-87348-6_5

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-87348-6_5

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-87347-9

  • Online ISBN: 978-3-030-87348-6

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation