Skip to main content
SpringerLink
Log in

Concurrent runtime verification of data rich events

  • General
  • Special Issue: RV 2020
  • Published:
International Journal on Software Tools for Technology Transfer Aims and scope Submit manuscript

Abstract

This paper presents the open-source runtime verification tool MESA (MEssage-based System Analysis), implemented in Scala, which supports concurrent monitors using the Actor model. Furthermore, the tool supports indexing (slicing) on the data values occurring in data-carrying events, for each individual monitor. The tool is generic in the sense that any monitoring system can be used for creating monitors. In this paper, we use the internal Scala DSL Daut for programming such data-parameterized state machines and temporal logic. To illustrate MESA/Daut, we present a case study that monitors flights from live U.S. airspace data streams, verifying that they conform to planned routes. With base in the case study, we then perform an extensive empirical study of the potential benefits from monitoring slices of a single property in concurrently executing actors. Due to the overhead of scheduling “small” actors (one for each slice or a small number of slices), it is not obvious that concurrent execution of such is beneficial. However, as a main result, we demonstrate that concurrent monitoring of slices to handle data-carrying events can provide considerable speed gains.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Aceto, L., Achilleos, A., Francalanza, A., Ingólfsdóttir, A., Lehtinen, K.: An operational guide to monitorability with applications to regular properties. Softw. Syst. Model. 20(2), 335–361 (2021). https://doi.org/10.1007/s10270-020-00860-z

    Article  Google Scholar 

  2. Aceto, L., Attard, D.P., Francalanza, A., Ingólfsdóttir, A.: On benchmarking for concurrent runtime verification. In: Guerra, E., Stoelinga, M. (eds.) Fundamental Approaches to Software Engineering, pp. 3–23. Springer, Berlin (2021). https://doi.org/10.1007/978-3-030-71500-7_1

    Chapter  Google Scholar 

  3. Agrawal, S., Bonakdarpour, B.: Runtime verification of k-safety hyperproperties in HyperLTL. In: 2016 IEEE 29th Computer Security Foundations Symposium (CSF), pp. 239–252 (2016). https://doi.org/10.1109/CSF.2016.24

    Chapter  Google Scholar 

  4. Akka (2020). http://doc.akka.io/docs/akka/current/scala.html

  5. Artho, C., Havelund, K., Kumar, R., Yamagata, Y.: Domain-specific languages with Scala. In: Butler, M., Conchon, S., Zaïdi, F. (eds.) Formal Methods and Software Engineering. Lecture Notes in Computer Science, vol. 9407, pp. 1–16. Springer, Berlin (2015). https://doi.org/10.1007/978-3-319-25423-4_1

    Chapter  Google Scholar 

  6. Attard, D.P., Francalanza, A.: Trace partitioning and local monitoring for asynchronous components. In: Cimatti, A., Sirjani, M. (eds.) International Conference on Software Engineering and Formal Methods. Lecture Notes in Computer Science, vol. 10469, pp. 219–235. Springer, Berlin (2017). https://doi.org/10.1007/978-3-319-66197-1_14

    Chapter  Google Scholar 

  7. Avrekh, I., Matthews, B.L., Stewart, M.: RNAV adherence data integration system using aviation and environmental sources. Tech. rep., NASA Ames Research Center (2018)

  8. Barre, B., Klein, M., Soucy-Boivin, M., Ollivier, P.A., Hallé, S.: MapReduce for parallel trace validation of LTL properties. In: Qadeer, S., Tasiran, S. (eds.) International Conference on Runtime Verification. Lecture Notes in Computer Science, vol. 7687, pp. 184–198. Springer, Berlin (2013). https://doi.org/10.1007/978-3-642-35632-2_20

    Chapter  Google Scholar 

  9. Barringer, H., Havelund, K.: TraceContract: a Scala DSL for trace analysis. In: Butler, M., Schulte, W. (eds.) International Symposium on Formal Methods. Lecture Notes in Computer Science, vol. 6664, pp. 57–72. Springer, Berlin (2011). https://doi.org/10.1007/978-3-642-21437-0_7

    Chapter  Google Scholar 

  10. Basin, D., Caronni, G., Ereth, S., Harvan, M., Klaedtke, F., Mantel, H.: Scalable offline monitoring of temporal specification. Form. Methods Syst. Des. 49, 75–108 (2016). https://doi.org/10.1007/s10703-016-0242-y

    Article  MATH  Google Scholar 

  11. Basin, D.A., Dardinier, T., Heimes, L., Krstic, S., Raszyk, M., Schneider, J., Traytel, D.: A formally verified, optimized monitor for metric first-order dynamic logic. In: Peltier, N., Sofronie-Stokkermans, V. (eds.) Automated Reasoning. Lecture Notes in Computer Science, vol. 12166, pp. 432–453. Springer, Berlin (2020). https://doi.org/10.1007/978-3-030-51074-9_25

    Chapter  Google Scholar 

  12. Basin, D., Gras, M., Krstić, S., Schneider, J.: Scalable online monitoring of distributed systems. In: Deshmukh, J., Nickovic, D. (eds.) Runtime Verification - 20th International Conference, RV 2020, Los Angeles, CA, USA, October 6–9, 2020. Lecture Notes in Computer Science vol. 12399, pp. 197–220. Springer, Berlin (2020). https://doi.org/10.1007/978-3-030-60508-7_11

    Chapter  Google Scholar 

  13. Berkovich, S., Bonakdarpour, B., Fischmeister, S.: Runtime verification with minimal intrusion through parallelism. Form. Methods Syst. Des. 46, 317–348 (2015). https://doi.org/10.1007/s10703-015-0226-3

    Article  MATH  Google Scholar 

  14. Burlò, C.B., Francalanza, A., Scalas, A.: On the monitorability of session types, in theory and practice (extended version). CoRR (2021). arXiv:2105.06291. https://doi.org/10.4230/LIPIcs.ECOOP.2021.20

  15. Clarkson, M.R., Schneider, F.B.: Hyperprop. J. Comput. Secur. 18(6), 1157–1210 (2010). https://doi.org/10.3233/JCS-2009-0393

    Article  Google Scholar 

  16. Colombo, C., Francalanza, A., Mizzi, R., Pace, G.J.: polyLarva: runtime verification with configurable resource-aware monitoring boundaries. In: Eleftherakis, G., Hinchey, M., Holcombe, M. (eds.) Software Engineering and Formal Methods. Lecture Notes in Computer Science, vol. 7504, pp. 218–232. Springer, Berlin (2012). https://doi.org/10.1007/978-3-642-33826-7_15

    Chapter  Google Scholar 

  17. Department of Transportation, Federal Aviation Administration: Implementation of Descend via into Boston Terminal area from Boston ARTCC (2015)

  18. El-Hokayem, A., Falcone, Y.: Can we monitor all multithreaded programs? In: Colombo, C., Leucker, M. (eds.) International Conference on Runtime Verification. Lecture Notes in Computer Science, vol. 11237, pp. 64–89. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-03769-7_6

    Chapter  Google Scholar 

  19. El-Hokayem, A., Falcone, Y.: On the monitoring of decentralized specifications: semantics, properties, analysis, and simulation. ACM Trans. Softw. Eng. Methodol. 29(1), 1:1–1:57 (2020). https://doi.org/10.1145/3355181

    Article  Google Scholar 

  20. Falcone, Y., Havelund, K., Reger, G.: A tutorial on runtime verification. In: Broy, M., Peled, D., Kalus, G. (eds.) Engineering Dependable Software Systems. NATO Science for Peace and Security Series - D: Information and Communication Security, vol. 34, pp. 141–175. IOS Press, Amsterdam (2013). https://doi.org/10.3233/978-1-61499-207-3-141

    Chapter  Google Scholar 

  21. Finkbeiner, B., Hahn, C., Stenger, M., Tentrup, L.: Monitoring hyperproperties. Form. Methods Syst. Des. 54(3), 336–363 (2019). https://doi.org/10.1007/s10703-019-00334-z

    Article  MATH  Google Scholar 

  22. Finkbeiner, B., Oswald, S., Passing, N., Schwenger, M.: Verified Rust monitors for Lola specifications. CoRR (2020). arXiv:2012.08961. https://doi.org/10.1007/978-3-030-60508-7_24

  23. Francalanza, A., Seychell, A.: Synthesising correct concurrent runtime monitors. Form. Methods Syst. Des. 46(3), 226–261 (2015). https://doi.org/10.1007/s10703-014-0217-9

    Article  MATH  Google Scholar 

  24. Francalanza, A., Pérez, J.A., Sánchez, C.: Runtime Verification for Decentralised and Distributed Systems pp. 176–210. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-75632-5_6

    Book  Google Scholar 

  25. Hallé, S., Khoury, R.: Event stream processing with BeepBeep 3. In: Reger, G., Havelund, K. (eds.) An International Workshop on Competitions, Usability, Benchmarks, Evaluation, and Standardisation for Runtime Verification Tools, Seattle, WA, USA, September 15, 2017. Kalpa Publications in Computing, vol. 3, pp. 81–88. EasyChair (2017). https://doi.org/10.29007/4cth

    Chapter  Google Scholar 

  26. Hallé, S., Khoury, R., Gaboury, R.: Event stream processing with multiple threads. In: Lahiri, S., Reger, G. (eds.) International Conference on Runtime Verification. Lecture Notes in Computer Science, vol. 10548, pp. 359–369. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-67531-2_22

    Chapter  Google Scholar 

  27. Harris Corporation: FAA Telecommunications Infrastructure NEMS User Guide (2013)

  28. Havelund, K.: Data automata in Scala. In: Symposium on Theoretical Aspects of Software Engineering Conference, Changsha, China, pp. 1–9 (2014). https://doi.org/10.1109/TASE.2014.37

    Chapter  Google Scholar 

  29. Havelund, K.: Daut (2022). https://github.com/havelund/daut

  30. Havelund, K.: TraceContract (2022). https://github.com/havelund/tracecontract

  31. Hewitt, C., Bishop, P., Steiger, R.: A universal modular ACTOR formalism for artificial intelligence. In: Proceedings of the 3rd International Joint Conference on Artificial Intelligence, pp. 235–245. Kaufmann, San Francisco (1973)

    Google Scholar 

  32. HOCON: Human Optimized Config Object Notation (2020). https://github.com/typesafehub/config/blob/master/HOCON.md

  33. International Air Line Pilots Associations: FAA Suspends OPD Arrivals for Atlanta International Airport (2016)

  34. International Civil Aviation Organization (ICAO): Performance-based Navigation (PBN) Manual, 3rd edn. (2008)

  35. JMH - Java Microbenchmark Harness (2020). https://openjdk.java.net/projects/code-tools/jmh/

  36. Joyce, J., Lomow, G., Slind, K., Unger, B.: Monitoring distributed systems. ACM Trans. Comput. Syst. 5(2), 121–150 (1987). https://doi.org/10.1145/13677.22723

    Article  Google Scholar 

  37. Kurklu, E., Havelund, K.: A flight rule checker for the LADEE Lunar spacecraft. In: Pun, V.K.I., Stolz, V., Simao, A. (eds.) Theoretical Aspects of Computing - ICTAC 2020, pp. 3–20. Springer, Berlin (2020). https://doi.org/10.1007/978-3-030-64276-1_1

    Chapter  Google Scholar 

  38. Lavery, P., Watanabe, T.: An actor-based runtime monitoring system for web and desktop applications. In: Hochin, T., Hirata, H., Nomiya, H. (eds.) International Conference on Software Engineering, Artificial Intelligence, Networking and Parallel/Distributed Computing, pp. 385–390. IEEE Comput. Soc., Los Alamitos (2017). https://doi.org/10.1109/SNPD.2017.8022750

    Chapter  Google Scholar 

  39. Leucker, M., Schallhart, C.: A brief account of runtime verification. J. Log. Algebraic Program. 78(5), 293–303 (2009). https://doi.org/10.1016/j.jlap.2008.08.004

    Article  MATH  Google Scholar 

  40. Mehlitz, P.: RACE (2022). http://nasarace.github.io/race/

  41. Mehlitz, P., Shafiei, N., Tkachuk, O., Davies, M.: RACE: building airspace simulations faster and better with actors. In: Digital Avionics Systems Conference (DASC), pp. 1–9 (2016). https://doi.org/10.1109/DASC.2016.7777991

    Chapter  Google Scholar 

  42. Meredith, P.O., Jin, D., Griffith, D., Chen, F., Roşu, G.: An overview of the MOP runtime verification framework. Int. J. Softw. Tools Technol. Transf. 14(3), 249–289 (2012). https://doi.org/10.1007/s10009-011-0198-6

    Article  Google Scholar 

  43. MESA - MEssage-based System Analysis (2022). https://github.com/NASA-SW-VnV/mesa

  44. NASA WorldWind (2022). https://worldwind.arc.nasa.gov/

  45. Neykova, R., Yoshida, N.: Let it recover: multiparty protocol-induced recovery. In: Wu, P., Hack, S. (eds.) International Conference on Compiler Construction, pp. 98–108. ACM, New York (2017). https://doi.org/10.1145/3033019.3033031

    Chapter  Google Scholar 

  46. Rasmussen, S., Kingston, D., Humphrey, L.: A brief introduction to unmanned systems autonomy services (UxAS). In: 2018 International Conference on Unmanned Aircraft Systems (ICUAS), pp. 257–268 (2018). https://doi.org/10.1109/ICUAS.2018.8453287

    Chapter  Google Scholar 

  47. Reger, G.: Rule-based runtime verification in a multicore system setting. Master’s thesis, University of Manchester (2010)

  48. Reger, G., Cruz, H.C., Rydeheard, D.: MarQ: monitoring at runtime with QEA. In: Baier, C., Tinelli, C. (eds.) International Conference on Tools and Algorithms for the Construction and Analysis of Systems. Lecture Notes in Computer Science, vol. 9035, pp. 596–610. Springer, Berlin (2015). https://doi.org/10.1007/978-3-662-46681-0_55

    Chapter  Google Scholar 

  49. Reger, G., Rydeheard, D., Barringer, H.: MAIL - an interaction layer for exploring the use of multicore in runtime monitoring. (unpublished)

  50. Richards, M., Monson-Haefel, R., Chappell, D.A.: Java Message Service, 2nd edn. O’Reilly Media, Inc., Newton (2009)

    Google Scholar 

  51. Roestenburg, R., Bakker, R., Williams, R.: Akka in Action, 1st edn. Manning Publications Co., Greenwich (2015)

    Google Scholar 

  52. Shafiei, N., Havelund, K., Mehlitz, P.: Actor-based runtime verification with MESA. In: Deshmukh, J., Ničković, D. (eds.) Runtime Verification, pp. 221–240. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-60508-7_12

    Chapter  Google Scholar 

  53. Shafiei, N., Havelund, K., Mehlitz, P.: Empirical Study of Actor-based Runtime Verification. Tech. rep., NASA Ames Research Center (2020)

  54. Stewart, M., Matthews, B.: Objective assessment method for RNAV STAR adherence. In: DASC: Digital Avionics Systems Conference (2017). https://doi.org/10.1109/DASC.2017.8102034

    Chapter  Google Scholar 

  55. SWIM flight data publication service (2020). https://www.faa.gov/air_traffic/technology/swim/sfdps/

  56. U.S. Department of Transportation. Federal Aviation Administration: Performance Based Navigation PBN NAS Navigation Strategy (2016)

  57. U.S. Department of Transportation. Federal Aviation Administration: Instrument Procedures Handbook (IPH) (2017)

Download references

Author information

Authors and Affiliations

  1. NASA Ames Research Center/KBR Inc., Moffett Field, CA, 94035, USA

    Nastaran Shafiei & Peter Mehlitz

  2. Jet Propulsion Laboratory, California Institute of Technology, Pasadena, CA, 91109, USA

    Klaus Havelund

Authors
  1. Nastaran Shafiei
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Klaus Havelund
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Peter Mehlitz
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Nastaran Shafiei.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Shafiei, N., Havelund, K. & Mehlitz, P. Concurrent runtime verification of data rich events. Int J Softw Tools Technol Transfer 25, 481–501 (2023). https://doi.org/10.1007/s10009-023-00713-2

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10009-023-00713-2

Keywords

Search

Navigation