Skip to main content
SpringerLink
Log in
SpringerLink
Log in

Towards Safe and Resilient Hybrid Systems in the Presence of Learning and Uncertainty

  • Conference paper
  • First Online:
Leveraging Applications of Formal Methods, Verification and Validation. Verification Principles (ISoLA 2022)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 13701))

Included in the following conference series:

Abstract

Intelligent hybrid systems pose a major challenge for the analysis of safety and resilience. Deductive formal verification methods for such systems typically use strong abstractions and focus only on the worst-case behavior. This often seriously impedes performance. Quantitative analyses based on simulations and statistical model checking, on the other hand, are much better suited to take performance properties into account, but generally do not provide guarantees for all possible behaviors. In this paper, we present a novel approach to combine deductive formal verification and quantitative analysis. Our approach enables us to construct provably safe and resilient systems, which still achieve certain performance levels with a statistical guarantee. We demonstrate the applicability of our approach with a case study of an intelligent water distribution system, which is resilient towards pump failures.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 79.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 99.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    https://www.uni-muenster.de/EmbSys/research/Simulink2dL.html.

  2. 2.

    Scheduler (U, C) obtains different probabilities in Tables 6 and 7, since those probabilities relate to different STL properties.

References

  1. Abate, A., Katoen, J.P., Lygeros, J., Prandini, M.: Approximate model checking of stochastic hybrid systems. Eur. J. Control. 16(6), 624–641 (2010)

    Article  MathSciNet  Google Scholar 

  2. Adelt, J., Liebrenz, T., Herber, P.: Formal verification of intelligent hybrid systems that are modeled with simulink and the reinforcement learning toolbox. In: Huisman, M., Păsăreanu, C., Zhan, N. (eds.) FM 2021. LNCS, vol. 13047, pp. 349–366. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-90870-6_19

    Chapter  Google Scholar 

  3. Alshiekh, M., Bloem, R., Ehlers, R., Könighofer, B., Niekum, S., Topcu, U.: Safe reinforcement learning via shielding. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 32 (2018)

    Google Scholar 

  4. Alur, R.: Formal verification of hybrid systems. In: ACM International Conference on Embedded Software (EMSOFT), pp. 273–278 (2011)

    Google Scholar 

  5. Araiza-Illan, D., Eder, K., Richards, A.: Formal verification of control systems’ properties with theorem proving. In: UKACC International Conference on Control (CONTROL), pp. 244–249. IEEE (2014)

    Google Scholar 

  6. Bertrand, N., et al.: Stochastic timed automata. Log. Methods Comput. Sci. 10(4) (2014)

    Google Scholar 

  7. Cai, M., Peng, H., Li, Z., Kan, Z.: Learning-based probabilistic LTL motion planning with environment and motion uncertainties. IEEE Trans. Autom. Control 66(5), 2386–2392 (2021)

    Article  MathSciNet  Google Scholar 

  8. Chen, M., et al.: MARS: a toolchain for modelling, analysis and verification of hybrid systems. In: Hinchey, M.G., Bowen, J.P., Olderog, E.-R. (eds.) Provably Correct Systems. NMSSE, pp. 39–58. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-48628-4_3

    Chapter  Google Scholar 

  9. Chutinan, A., Krogh, B.H.: Computational techniques for hybrid system verification. IEEE Trans. Autom. Control 48(1), 64–75 (2003)

    Article  MathSciNet  Google Scholar 

  10. Ellen, C., Gerwinn, S., Fränzle, M.: Statistical model checking for stochastic hybrid systems involving nondeterminism over continuous domains. Int. J. Softw. Tools Technol. Transf. 17(4), 485–504 (2015)

    Article  Google Scholar 

  11. Fulton, N., Hunt, N., Hoang, N., Das, S.: Formal Verification of End-to-End Learning in Cyber-Physical Systems: Progress and Challenges. arXiv:2006.09181 (2020)

  12. Fulton, N., Mitsch, S., Quesel, J.-D., Völp, M., Platzer, A.: KeYmaera X: an axiomatic tactical theorem prover for hybrid systems. In: Felty, A.P., Middeldorp, A. (eds.) CADE 2015. LNCS (LNAI), vol. 9195, pp. 527–538. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-21401-6_36

    Chapter  Google Scholar 

  13. Fulton, N., Platzer, A.: Safe reinforcement learning via formal methods: toward safe control through proof and learning. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 32 (2018)

    Google Scholar 

  14. Gribaudo, M., Remke, A.: Hybrid Petri nets with general one-shot transitions. Perform. Eval. 105, 22–50 (2016)

    Article  Google Scholar 

  15. Gudemann, M., Ortmeier, F.: A framework for qualitative and quantitative formal model-based safety analysis. In: IEEE International Symposium on High Assurance Systems Engineering, pp. 132–141. IEEE (2010)

    Google Scholar 

  16. Hahn, E.M., Hartmanns, A., Hermanns, H., Katoen, J.P.: A compositional modelling and analysis framework for stochastic hybrid systems. Form. Methods Syst. Des. 43(2), 191–232 (2013)

    Article  Google Scholar 

  17. Hahn, E.M., Perez, M., Schewe, S., Somenzi, F., Trivedi, A., Wojtczak, D.: Faithful and effective reward schemes for model-free reinforcement learning of omega-regular objectives. In: Hung, D.V., Sokolsky, O. (eds.) ATVA 2020. LNCS, vol. 12302, pp. 108–124. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-59152-6_6

    Chapter  Google Scholar 

  18. Hasanbeig, M., Kantaros, Y., Abate, A., Kroening, D., Pappas, G.J., Lee, I.: Reinforcement learning for temporal logic control synthesis with probabilistic satisfaction guarantees. In: IEEE Conference on Decision and Control (CDC), Nice, France, pp. 5338–5343. IEEE (2019)

    Google Scholar 

  19. Hasanbeig, M., Abate, A., Kroening, D.: Cautious reinforcement learning with logical constraints. In: International Foundation for Autonomous Agents and Multiagent Systems, AAMAS 2020, pp. 483–491 (2020)

    Google Scholar 

  20. Herber, P., Reicherdt, R., Bittner, P.: Bit-precise formal verification of discrete-time MATLAB/Simulink models using SMT solving. In: International Conference on Embedded Software (EMSOFT), pp. 1–10. IEEE (2013)

    Google Scholar 

  21. Junges, S., Jansen, N., Katoen, J.-P., Topcu, U., Zhang, R., Hayhoe, M.: Model checking for safe navigation among humans. In: McIver, A., Horvath, A. (eds.) QEST 2018. LNCS, vol. 11024, pp. 207–222. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-99154-2_13

    Chapter  Google Scholar 

  22. Knüppel, A., Thüm, T., Schaefer, I.: GUIDO: automated guidance for the configuration of deductive program verifiers. In: IEEE/ACM International Conference on Formal Methods in Software Engineering (FormaliSE), pp. 124–129. IEEE (2021)

    Google Scholar 

  23. Könighofer, B., Lorber, F., Jansen, N., Bloem, R.: Shield synthesis for reinforcement learning. In: Margaria, T., Steffen, B. (eds.) ISoLA 2020. LNCS, vol. 12476, pp. 290–306. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-61362-4_16

    Chapter  Google Scholar 

  24. Laprie, J.C.: From dependability to resilience. In: IEEE/IFIP International Conference on Dependable Systems and Networks (DSN), pp. G8–G9 (2008)

    Google Scholar 

  25. Liebrenz, T., Herber, P., Glesner, S.: Deductive verification of hybrid control systems modeled in simulink with KeYmaera X. In: Sun, J., Sun, M. (eds.) ICFEM 2018. LNCS, vol. 11232, pp. 89–105. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-02450-5_6

    Chapter  Google Scholar 

  26. Liebrenz, T., Herber, P., Glesner, S.: A service-oriented approach for decomposing and verifying hybrid system models. In: Arbab, F., Jongmans, S.-S. (eds.) FACS 2019. LNCS, vol. 12018, pp. 127–146. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-40914-2_7

    Chapter  Google Scholar 

  27. Liebrenz, T., Herber, P., Glesner, S.: Service-oriented decomposition and verification of hybrid system models using feature models and contracts. Sci. Comput. Program. 211, 102694 (2021)

    Article  Google Scholar 

  28. Lygeros, J., Prandini, M.: Stochastic hybrid systems: a powerful framework for complex, large scale applications. Eur. J. Control. 16(6), 583–594 (2010)

    Article  MathSciNet  Google Scholar 

  29. Maler, O., Nickovic, D.: Monitoring temporal properties of continuous signals. In: Lakhnech, Y., Yovine, S. (eds.) FORMATS/FTRTFT -2004. LNCS, vol. 3253, pp. 152–166. Springer, Heidelberg (2004). https://doi.org/10.1007/978-3-540-30206-3_12

    Chapter  MATH  Google Scholar 

  30. Minopoli, S., Frehse, G.: SL2SX translator: from Simulink to SpaceEx models. In: International Conference on Hybrid Systems: Computation and Control, pp. 93–98. ACM (2016)

    Google Scholar 

  31. Niehage, M., Hartmanns, A., Remke, A.: Learning optimal decisions for stochastic hybrid systems. In: ACM-IEEE International Conference on Formal Methods and Models for System Design (MEMOCODE), pp. 44–55. ACM (2021)

    Google Scholar 

  32. Pilch, C., Edenfeld, F., Remke, A.: HYPEG: statistical model checking for hybrid petri nets: tool paper. In: EAI International Conference on Performance Evaluation Methodologies and Tools (VALUETOOLS), pp. 186–191. ACM Press (2017)

    Google Scholar 

  33. Pilch, C., Niehage, M., Remke, A.: HPnGs go non-linear: statistical dependability evaluation of battery-powered systems. In: IEEE International Symposium on Modeling, Analysis, and Simulation of Computer and Telecommunication Systems (MASCOTS), pp. 157–169. IEEE (2018)

    Google Scholar 

  34. Pilch, C., Remke, A.: Statistical model checking for hybrid petri nets with multiple general transitions. In: Annual IEEE/IFIP International Conference on Dependable Systems and Networks (DSN), pp. 475–486. IEEE (2017)

    Google Scholar 

  35. Platzer, A.: Differential dynamic logic for hybrid systems. J. Autom. Reason. 41(2), 143–189 (2008)

    Article  MathSciNet  Google Scholar 

  36. Reicherdt, R., Glesner, S.: Formal verification of discrete-time MATLAB/Simulink models using boogie. In: Giannakopoulou, D., Salaün, G. (eds.) SEFM 2014. LNCS, vol. 8702, pp. 190–204. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-10431-7_14

    Chapter  Google Scholar 

  37. Sadigh, D., Kim, E.S., Coogan, S., Sastry, S.S., Seshia, S.A.: A learning based approach to control synthesis of Markov decision processes for linear temporal logic specifications. In: IEEE Conference on Decision and Control, pp. 1091–1096. IEEE (2014)

    Google Scholar 

  38. Shmarov, F., Zuliani, P.: Probabilistic hybrid systems verification via SMT and Monte Carlo techniques. In: Bloem, R., Arbel, E. (eds.) HVC 2016. LNCS, vol. 10028, pp. 152–168. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-49052-6_10

    Chapter  Google Scholar 

  39. Sutton, R.S., Barto, A.G.: Reinforcement Learning: An Introduction, 2nd edn. The MIT Press, Cambridge; London (2018)

    MATH  Google Scholar 

  40. The MathWorks: Simulink. https://de.mathworks.com/products/simulink.html

  41. The MathWorks: Reinforcement Learning Toolbox. https://www.mathworks.com/products/reinforcement-learning.html

  42. The MathWorks: Simulink Design Verifier. https://de.mathworks.com/products/simulink-design-verifier.html

  43. The MathWorks: Simulink Example: Water Distribution System Scheduling Using Reinforcement Learning. https://de.mathworks.com/help/reinforcement-learning/ug/water-distribution-scheduling-system.html

  44. Zou, L., Zhan, N., Wang, S., Fränzle, M.: Formal verification of simulink/stateflow diagrams. In: Finkbeiner, B., Pu, G., Zhang, L. (eds.) ATVA 2015. LNCS, vol. 9364, pp. 464–481. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-24953-7_33

    Chapter  MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. University of Münster, Einsteinstr. 62, 48149, Münster, Germany

    Julius Adelt, Paula Herber, Mathis Niehage & Anne Remke

Authors
  1. Julius Adelt
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Paula Herber
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Mathis Niehage
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Anne Remke
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Paula Herber .

Editor information

Editors and Affiliations

  1. University of Limerick, CSIS and Lero, Limerick, Ireland

    Tiziana Margaria

  2. TU Dortmund, Dortmund, Germany

    Bernhard Steffen

Rights and permissions

Reprints and permissions

Copyright information

© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Adelt, J., Herber, P., Niehage, M., Remke, A. (2022). Towards Safe and Resilient Hybrid Systems in the Presence of Learning and Uncertainty. In: Margaria, T., Steffen, B. (eds) Leveraging Applications of Formal Methods, Verification and Validation. Verification Principles. ISoLA 2022. Lecture Notes in Computer Science, vol 13701. Springer, Cham. https://doi.org/10.1007/978-3-031-19849-6_18

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-19849-6_18

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-19848-9

  • Online ISBN: 978-3-031-19849-6

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation