Skip to main content
SpringerLink
Log in

Formal Specification for Learning-Enabled Autonomous Systems

  • Conference paper
  • First Online:
Software Verification and Formal Methods for ML-Enabled Autonomous Systems (NSV 2022, FoMLAS 2022)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 13466))

Abstract

The formal specification provides a uniquely readable description of various aspects of a system, including its temporal behavior. This facilitates testing and sometimes automatic verification of the system against the given specification. We present a logic-based formalism for specifying learning-enabled autonomous systems, which involve components based on neural networks. The formalism is based on first-order past time temporal logic that uses predicates for denoting events. We have applied the formalism successfully to two complex use cases.

Supported by the european project Horizon 2020 research and innovation programme under grant agreement No. 956123.

C.-H. Cheng—The work is primarily conducted during his service at DENSO.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 54.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 69.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    http://www.foceta-project.eu/.

  2. 2.

    \(\gamma \, [ x \mapsto a ]\) is the overriding of \(\gamma \) with the binding \([ x \mapsto a ]\).

References

  1. Alpern, B., Schneider, F.B.: Recognizing safety and liveness. Distrib. Comput. 2(3), 117–126 (1987)

    Article  MATH  Google Scholar 

  2. Alshiekh, M., Bloem, R., Ehlers, R., Könighofer, B., Niekum, S., Topcu, U.: Safe reinforcement learning via shielding. In: AAAI 2018, pp. 2669–2678 (2018)

    Google Scholar 

  3. Apt, K.R., Kozen, D.: Limits for automatic verification of finite-state concurrent systems. Inf. Process. Lett. 22(6), 307–309 (1986)

    Article  MathSciNet  Google Scholar 

  4. Balakrishnan, A., et al.: Specifying and evaluating quality metrics for vision-based perception systems. In: DATE, pp. 1433–1438 (2019)

    Google Scholar 

  5. Bartocci, E., Bloem, R., Maderbacher, B., Manjunath, N., Nickovic, D.: Adaptive testing for CPS with specification coverage. In: ADHS 2021 (2021)

    Google Scholar 

  6. Bartocci, E., Falcone, Y., Francalanza, A., Reger, G.: Introduction to runtime verification. In: Bartocci, E., Falcone, Y. (eds.) Lectures on Runtime Verification. LNCS, vol. 10457, pp. 1–33. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-75632-5_1

    Chapter  Google Scholar 

  7. Basin, D.A., Klaedtke, F., Müller, S., Zalinescu, E.: Monitoring metric first-order temporal properties. J. ACM 62(2), 45 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  8. Bauer, A., Leucker, M., Schallhart, C.: The good, the bad, and the ugly, but how ugly is ugly? In: Sokolsky, O., Taşıran, S. (eds.) RV 2007. LNCS, vol. 4839, pp. 126–138. Springer, Heidelberg (2007). https://doi.org/10.1007/978-3-540-77395-5_11

    Chapter  MATH  Google Scholar 

  9. Bloem, R., et al.: RATSY – a new requirements analysis tool with synthesis. In: Touili, T., Cook, B., Jackson, P. (eds.) CAV 2010. LNCS, vol. 6174, pp. 425–429. Springer, Heidelberg (2010). https://doi.org/10.1007/978-3-642-14295-6_37

    Chapter  Google Scholar 

  10. Clarke, E.M., Emerson, E.A.: Design and synthesis of synchronization skeletons using branching time temporal logic. In: Kozen, D. (ed.) Logic of Programs 1981. LNCS, vol. 131, pp. 52–71. Springer, Heidelberg (1982). https://doi.org/10.1007/BFb0025774

    Chapter  Google Scholar 

  11. Cordts, M., et al.: The cityscapes dataset for semantic urban scene understanding. CoRR, abs/1604.01685 (2016)

    Google Scholar 

  12. Donzé, A., Maler, O.: Robust satisfaction of temporal logic over real-valued signals. In: Chatterjee, K., Henzinger, T.A. (eds.) FORMATS 2010. LNCS, vol. 6246, pp. 92–106. Springer, Heidelberg (2010). https://doi.org/10.1007/978-3-642-15297-9_9

    Chapter  MATH  Google Scholar 

  13. Fainekos, G.E., Pappas, G.J.: Robustness of temporal logic specifications. In: Havelund, K., Núñez, M., Roşu, G., Wolff, B. (eds.) FATES/RV 2006. LNCS, vol. 4262, pp. 178–192. Springer, Heidelberg (2006). https://doi.org/10.1007/11940197_12

    Chapter  Google Scholar 

  14. Falcone, Y., Mounier, L., Fernandez, J.-C., Richier, J.-L.: Runtime enforcement monitors: composition, synthesis, and enforcement abilities. Formal Methods Syst. Des. 38(3), 223–262 (2011)

    Article  MATH  Google Scholar 

  15. Ferrère, T., Nickovic, D., Donzé, A., Ito, H., Kapinski, J.: Interface-aware signal temporal logic. In: HSCC 2019, pp. 57–66 (2019)

    Google Scholar 

  16. Fowler, M., Distilled, U.M.L.: A Brief Guide to the Standard Object Modeling Language. Addison-Wesley, Boston (2004)

    Google Scholar 

  17. Havelund, K., Peled, D., Ulus, D.: First order temporal logic monitoring with BDDs. In: FMCAD 2017, pp. 116–123 (2017)

    Google Scholar 

  18. Hong, H.S., Lee, I., Sokolsky, O., Ural, H.: A temporal logic based theory of test coverage and generation. In: Katoen, J.-P., Stevens, P. (eds.) TACAS 2002. LNCS, vol. 2280, pp. 327–341. Springer, Heidelberg (2002). https://doi.org/10.1007/3-540-46002-0_23

    Chapter  MATH  Google Scholar 

  19. Manna, Z., Pnueli, A.: Completing the temporal picture. Theor. Comput. Sci. 83, 91–130 (1991)

    Article  MATH  Google Scholar 

  20. Nghiem, T., Sankaranarayanan, S., Fainekos, G., Ivancic, F., Gupta, A., Pappas, G.: Monte-Carlo techniques for falsification of temporal properties of non-linear hybrid systems. In: HSCC 2010, pp. 211–220 (2010)

    Google Scholar 

  21. Queille, J.P., Sifakis, J.: Specification and verification of concurrent systems in cesar. In: Dezani-Ciancaglini, M., Montanari, U. (eds.) Programming 1982. LNCS, vol. 137, pp. 337–351. Springer, Heidelberg (1982). https://doi.org/10.1007/3-540-11494-7_22

    Chapter  Google Scholar 

  22. Prabhakar, P., Lal, R., Kapinski, J.: Automatic trace generation for signal temporal logic. In: RTSS 2018, pp. 208–217 (2018)

    Google Scholar 

  23. Redmon, J., Divvala, S., Girshick, R., Farhadi, A.: You only look once: unified, real-time object detection. In: CVPR 2016, pp. 779–788 (2016)

    Google Scholar 

  24. Roehm, H., Heinz, T., Mayer, E.C.: STLInspector: STL validation with guarantees. In: Majumdar, R., Kunčak, V. (eds.) CAV 2017, Part I. LNCS, vol. 10426, pp. 225–232. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-63387-9_11

    Chapter  Google Scholar 

  25. Beckert, B., Hähnle, R., Schmitt, P.H. (eds.): Verification of Object-Oriented Software. The KeY Approach. LNCS (LNAI), vol. 4334. Springer, Heidelberg (2007). https://doi.org/10.1007/978-3-540-69061-0

    Book  Google Scholar 

  26. Smullyan, R.R.: First-Order Logic. Ergebnisse der Mathematik und ihrer Grenzgebiete. 2. Folge, Springer, Heidelberg (2012). https://doi.org/10.1007/978-3-642-86718-7

    Book  MATH  Google Scholar 

  27. Balakrishnan, A., Deshmukh, J., Hoxha, B., Yamaguchi, T., Fainekos, G.: PerceMon: online monitoring for perception systems. In: Feng, L., Fisman, D. (eds.) RV 2021. LNCS, vol. 12974, pp. 297–308. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-88494-9_18

    Chapter  Google Scholar 

  28. Dutle, A., et al: From requirements to autonomous flight: an overview of the monitoring ICAROUS project. In: Proceedings of 2nd Workshop on Formal Methods for Autonomous Systems (FMAS). EPTCS, vol. 329, pp. 23–30 (2020)

    Google Scholar 

  29. Koymans, R.: Specifying real-time properties with metric temporal logic. Real-Time Syst.. 2(4), 255–299 (1990)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. University Grenoble Alpes, VERIMAG, Grenoble, France

    Saddek Bensalem

  2. Fraunhofer IKS, Munich, Germany

    Chih-Hong Cheng

  3. University of Liverpool, Liverpool, L69 3BX, UK

    Xiaowei Huang

  4. Aristotle University of Thessaloniki, Thessaloniki, Greece

    Panagiotis Katsaros

  5. DENSO AUTOMOTIVE Deutschland GmbH, Eching, Germany

    Adam Molin

  6. AIT Austrian Institute of Technology, Vienna, Austria

    Dejan Nickovic

  7. Bar Ilan University, Ramat Gan, Israel

    Doron Peled

Authors
  1. Saddek Bensalem
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Chih-Hong Cheng
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Xiaowei Huang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Panagiotis Katsaros
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Adam Molin
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Dejan Nickovic
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Doron Peled
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Panagiotis Katsaros .

Editor information

Editors and Affiliations

  1. The Hebrew University of Jerusalem, Jerusalem, Israel

    Omri Isac

  2. Rensselaer Polytechnic Institute, New York, NY, USA

    Radoslav Ivanov

  3. The Hebrew University of Jerusalem, Jerusalem, Israel

    Guy Katz

  4. Palo Alto, CA, USA

    Nina Narodytska

  5. University of Trieste, Trieste, Italy

    Laura Nenzi

Rights and permissions

Reprints and permissions

Copyright information

© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Bensalem, S. et al. (2022). Formal Specification for Learning-Enabled Autonomous Systems. In: Isac, O., Ivanov, R., Katz, G., Narodytska, N., Nenzi, L. (eds) Software Verification and Formal Methods for ML-Enabled Autonomous Systems. NSV FoMLAS 2022 2022. Lecture Notes in Computer Science, vol 13466. Springer, Cham. https://doi.org/10.1007/978-3-031-21222-2_8

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-21222-2_8

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-21221-5

  • Online ISBN: 978-3-031-21222-2

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation