Skip to main content

LLM-Based Scheme for Synthesis of Formal Verification Algorithms

  • Conference paper
  • First Online:
Bridging the Gap Between AI and Reality (AISoLA 2024)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 15217))

Included in the following conference series:

  • 86 Accesses

Abstract

The research of Large Language Models (LLMs) has significant ground to cover in the context of formal verification. In this work, we present a methodology that aims to increase the reliability of code synthesized through the use of LLMs. Our approach capitalizes on the intrinsic knowledge embedded within LLMs to achieve a more reliable code synthesis. We specifically illustrate the possibility of teaching model checking and runtime verification (RV) algorithms through our approach. Our experiments demonstrate that LLMs grasp the concept of dynamic programming, allowing them to synthesize code for these verification tasks with minimal guidance.

This research was supported by the Israel Science Foundation grant “Validating and controlling software and hardware systems assisted by machine learning” No. 2454/23.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

Notes

  1. 1.

    https://github.com/albertocasagrande/pyModelChecking.

References

  1. Alpern, B., Schneider, F.B.: Recognizing safety and liveness. Distrib. Comput. 2(3), 117–126 (1987)

    Article  MATH  Google Scholar 

  2. Anil, R., et al.: Gemini: a family of highly capable multimodal models. CoRR arxiv:2312.11805 (2023)

  3. Baier, C., Katoen, J.P.: Principles of Model Checking. MIT press, Cambridge (2008)

    MATH  Google Scholar 

  4. Bellman, R.: Dynamic programming. Science 153(3731), 34–37 (1966)

    Article  MATH  Google Scholar 

  5. Clarke, E.M., Emerson, E.A.: Design and synthesis of synchronization skeletons using branching time temporal logic. In: Workshop on Logic of Programs, pp. 52–71. Springer, Heidelberg (1981). https://doi.org/10.1007/bfb0025774

  6. Clarke, E.M., Emerson, E.A.: Design and synthesis of synchronization skeletons using branching time temporal logic. In: Workshop on logic of programs. pp. 52–71. Springer (1981)

    Google Scholar 

  7. Cohen, I., Peled, D.: End-to-end AI generated runtime verification from natural language specification. In: Steffen, B. (ed.) International Conference on Bridging the Gap between AI and Reality, AISoLA 2023. LNCS, vol. 14129, pp. 362–384. Springer, Cham (2023). https://doi.org/10.1007/978-3-031-73741-1_23

    Chapter  MATH  Google Scholar 

  8. Grosu, R., Peled, D., Ramakrishnan, C.R., Smolka, S.A., Stoller, S.D., Yang, J.: Compositional Branching-Time Measurements. In: Bensalem, S., Lakhneck, Y., Legay, A. (eds.) ETAPS 2014. LNCS, vol. 8415, pp. 118–128. Springer, Heidelberg (2014). https://doi.org/10.1007/978-3-642-54848-2_8

  9. Havelund, K., Roşu, G.: Synthesizing monitors for safety properties. In: Katoen, J.P., Stevens, P. (eds.) Tools and Algorithms for the Construction and Analysis of Systems, pp. 342–356. Springer, Heidelberg (2002). https://doi.org/10.1007/3-540-46002-0_24

  10. Pnueli, A., Rosner, R.: On the synthesis of a reactive module. In: Conference Record of the Sixteenth Annual ACM Symposium on Principles of Programming Languages, Austin, Texas, USA, 11–13 January 1989, pp. 179–190. ACM Press (1989)

    Google Scholar 

  11. Pnueli, A., Rosner, R.: On the synthesis of an asynchronous reactive module. In: Ausiello, G., Dezani-Ciancaglini, M., Della Rocca, S.R. (eds.) ICALP 1989. LNCS, vol. 372, pp. 652–671. Springer, Heidelberg (1989). https://doi.org/10.1007/BFb0035790

  12. Queille, J.P., Sifakis, J.: Specification and verification of concurrent systems in cesar. In: International Symposium on Programming, pp. 337–351. Springer, Heidelberg (1982). https://doi.org/10.1007/3-540-11494-7_22

  13. Radford, A., Narasimhan, K., Salimans, T., Sutskever, I., et al.: Improving language understanding by generative pre-training. OpenAI (2018)

    Google Scholar 

  14. Sutton, R.S., Barto, A.G.: Reinforcement Learning: An Introduction. MIT press, Cambridge (2018)

    MATH  Google Scholar 

  15. Tan, C.W., Guo, S., Wong, M., Hang, C.N.: Copilot for xcode: exploring AI-assisted programming by prompting cloud-based large language models. CoRR arxiv:2307.14349 (2023)

Download references

Acknowledgement

We thank Moran Omer for carefully reading and providing helpful comments on an early draft of this paper.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Doron Peled .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and permissions

Copyright information

© 2025 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Cohen, I., Peled, D. (2025). LLM-Based Scheme for Synthesis of Formal Verification Algorithms. In: Steffen, B. (eds) Bridging the Gap Between AI and Reality. AISoLA 2024. Lecture Notes in Computer Science, vol 15217. Springer, Cham. https://doi.org/10.1007/978-3-031-75434-0_11

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-75434-0_11

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-75433-3

  • Online ISBN: 978-3-031-75434-0

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics