Abstract
This article presents a benchmark generator, Timescales, which can be used to evaluate the performance and scalability of runtime verification tools using Metric Temporal Logic (MTL) formulas as their specifications. We mainly target runtime verification of cyber-physical systems and generate traces similar to the qualitative behavior of sensor readings and state variables of such systems that are observed/sampled continuously. Since such systems are composed of many heterogeneous components that work over very different time scales, it is crucial to measure the performance of the MTL monitoring tool for a wide range of timing parameters in specifications. Hence, Timescales supports the generation of benchmarks for 10 typical timed properties for any given trace length and timing parameters with several other useful features. Finally, we include some default benchmark suites generated by Timescales.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
CSV is a common text-based data exchange format to store a sequence of data fields. (https://en.wikipedia.org/wiki/Comma-separated_values).
- 2.
YAML is a human-readable configuration file format. (https://yaml.org).
- 3.
ANTLR is a powerful parser generator tool. (https://www.antlr.org).
- 4.
References
Basin, D., Klaedtke, F., Zalinescu, E.: The MonPoly monitoring tool. In: Proceedings of the Workshop on Competitions, Usability, Benchmarks, Evaluation, and Standardisation for Runtime Verification Tools (RV-CuBES), vol. 3, pp. 19–28 (2017)
Basin, D., Krstic, S., Traytel, D.: AERIAL: almost event-rate independent algorithms for monitoring metric regular properties. In: Proceedings of the Workshop on Competitions, Usability, Benchmarks, Evaluation, and Standardisation for Runtime Verification Tools (RV-CuBES), pp. 29–36 (2017)
Dwyer, M.B. Avrunin, G.S., Corbett, J.C.: Property specification patterns for finite-state verification. In: Proceedings of the Workshop on Formal Methods in Software Practice (FMSP), pp. 7–15 (1998)
Gruhn, V., Laue, R.: Patterns for timed property specifications. Electron. Notes Theor. Comput. Sci. 153(2), 117–133 (2006)
Konrad, S., Cheng, B.H.C.: Real-time specification patterns. In: Proceedings of the International Conference on Software Engineering (ICSE), pp. 372–381 (2005)
Koymans, R.: Specifying real-time properties with metric temporal logic. Real-Time Syst. 2(4), 255–299 (1990)
Ničković, D., Lebeltel, O., Maler, O., Ferrère, T., Ulus, D.: AMT 2.0: qualitative and quantitative trace analysis with extended signal temporal logic. In: Proceedings of the Conference on Tools and Algorithms for the Construction and Analysis of Systems (TACAS), pp. 303–319 (2018)
Schumann, J., Moosbrugger, P., Rozier, K.Y.: R2U2: monitoring and diagnosis of security threats for unmanned aerial systems. In: Bartocci, E., Majumdar, R. (eds.) RV 2015. LNCS, vol. 9333, pp. 233–249. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-23820-3_15
Ulus, D.: Online monitoring of metric temporal logic using sequential networks. In: arXiv preprint arXiv:1901.00175 (2019)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
Ulus, D. (2019). Timescales: A Benchmark Generator for MTL Monitoring Tools. In: Finkbeiner, B., Mariani, L. (eds) Runtime Verification. RV 2019. Lecture Notes in Computer Science(), vol 11757. Springer, Cham. https://doi.org/10.1007/978-3-030-32079-9_25
Download citation
DOI: https://doi.org/10.1007/978-3-030-32079-9_25
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-32078-2
Online ISBN: 978-3-030-32079-9
eBook Packages: Computer ScienceComputer Science (R0)