Skip to main content
SpringerLink
Log in

Verifying verified code

  • S.I.: ATVA 2021
  • Published:
Innovations in Systems and Software Engineering Aims and scope Submit manuscript

Abstract

A recent case study from AWS by Chong et al. proposes an effective methodology for Bounded Model Checking in industry. In this paper, we report on a follow-up case study that explores the methodology from the perspective of three research questions: (a) can proof artefacts be used across verification tools; (b) are there bugs in verified code; and (c) can specifications be improved. To study these questions, we port the verification tasks for aws-c-common library to SeaHorn, SMACK and KLEE. We show the benefits of using compiler semantics and cross-checking specifications with different verification techniques, and call for standardizing proof library extensions to increase specification reuse. The verification tasks discussed are publicly available online.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

Notes

  1. By continuous verification, we mean verification that is integrated with continuous integration (CI) and is checked during every commit.

  2. https://github.com/awslabs/aws-c-common/tree/main/verification/cbmc.

  3. https://github.com/seahorn/verify-c-common.

  4. In Chong et al., these are called proof harnesses.

  5. https://seahorn.github.io/verify-c-common/fuzzing_coverage/index.html.

  6. Similarly, we introduced to replace lines 2–5 in Fig. 1.

  7. https://github.com/awslabs/aws-c-common/pull/686.

  8. Unit proof in .

References

  1. Rakamaric Z, Emmi M (2014) SMACK: decoupling source language details from verifier implementations. In: Computer aided verification—26th international conference, CAV 2014, held as part of the Vienna Summer of Logic, VSL 2014, Vienna, Austria, July 18-22, 2014. Proceedings. Lecture notes in computer science, vol 8559, pp 106–113

  2. Beyer D, Keremoglu ME (2011) CPAchecker: a tool for configurable software verification. In: Computer aided verification—23rd international conference, CAV 2011, Snowbird, UT, USA, July 14–20, 2011. Proceedings. Lecture notes in computer science, vol 6806, pp 184–190

  3. Gadelha MYR, Monteiro FR, Morse J, Cordeiro LC, Fischer B, Nicole DA (2018) ESBMC 5.0: an industrial-strength C model checker. In: Proceedings of the 33rd ACM/IEEE international conference on automated software engineering, ASE 2018, Montpellier, France, September 3–7, 2018, pp 888–891

  4. Lal A, Qadeer S (2014) Powering the static driver verifier using Corral. In: Proceedings of the 22nd ACM SIGSOFT international symposium on Foundations of Software Engineering, (FSE-22), Hong Kong, China, November 16–22, 2014, pp 202–212

  5. Ivancic F, Yang Z, Ganai MK, Gupta A, Shlyakhter I, Ashar P (2005) F-Soft: software verification platform. In: Computer aided verification, 17th international conference, CAV 2005, Edinburgh, Scotland, UK, July 6–10, 2005, Proceedings. Lecture notes in computer science, vol 3576, pp 301–306

  6. Clarke EM, Kroening D, Lerda F (2004) A tool for checking ANSI-C programs. In: Tools and algorithms for the construction and analysis of systems, 10th international conference, TACAS 2004, held as part of the Joint European Conferences on Theory and Practice of Software, ETAPS 2004, Barcelona, Spain, March 29–April 2, 2004, Proceedings. Lecture notes in computer science, vol 2988, pp 168–176

  7. Galois: Crux: a tool for improving the assurance of software using symbolic testing. https://crux.galois.com/

  8. Büning MK, Sinz C, Faragó D (2020) QPR verify: a static analysis tool for embedded software based on bounded model checking. In: Software verification—12th international conference, VSTTE 2020, and 13th international workshop, NSV 2020, Los Angeles, CA, USA, July 20–21, 2020, revised selected papers. Lecture notes in computer science, vol 12549, pp 21–32

  9. Beyer D (2020) Advances in automatic software verification: SV-COMP 2020. In: Tools and algorithms for the construction and analysis of systems—26th international conference, TACAS 2020, held as part of the European Joint Conferences on Theory and Practice of Software, ETAPS 2020, Dublin, Ireland, April 25–30, 2020, Proceedings, Part II. Lecture notes in computer science, vol 12079, pp 347–367

  10. Serebryany K.: libFuzzer: a library for coverage-guided fuzz testing. https://llvm.org/docs/LibFuzzer.html

  11. Bessey A, Block K, Chelf B, Chou A, Fulton B, Hallem S, Gros C, Kamsky A, McPeak S, Engler DR (2010) A few billion lines of code later: using static analysis to find bugs in the real world. Commun ACM 53(2):66–75

    Article  Google Scholar 

  12. Chong N, Cook B, Kallas K, Khazem K, Monteiro FR, Schwartz-Narbonne D, Tasiran S, Tautschnig M, Tuttle MR (2020) Code-level model checking in the software development workflow. In: ICSE-SEIP 2020: 42nd international conference on software engineering, software engineering in practice, Seoul, South Korea, 27 June–19 July, 2020, pp 11–20

  13. Gurfinkel A, Kahsai T, Komuravelli A, Navas JA (2015) The SeaHorn verification framework. In: Computer aided verification—27th international conference, CAV 2015, San Francisco, CA, USA, July 18–24, 2015, Proceedings, Part I. Lecture notes in computer science, vol 9206, pp 343–361

  14. Cadar C, Dunbar D, Engler DR (2008) KLEE: unassisted and automatic generation of high-coverage tests for complex systems programs. In: 8th USENIX symposium on operating systems design and implementation, OSDI 2008, December 8–10, 2008, San Diego, California, USA, Proceedings, pp 209–224

  15. Osherove R (2009) The art of unit testing: with examples in .Net

  16. Lattner C, Adve VS (2004) LLVM: a compilation framework for lifelong program analysis & transformation. In: 2nd IEEE/ACM international symposium on code generation and optimization (CGO 2004), 20–24 March 2004, San Jose, CA, USA, pp 75–88

  17. uClibc is a small C standard library. https://www.uclibc.org/

  18. Memarian K, Matthiesen J, Lingard J, Nienhuis K, Chisnall D, Watson RNM, Sewell P (2016) Into the depths of C: elaborating the de facto standards. In: Proceedings of the 37th ACM SIGPLAN conference on programming language design and implementation, PLDI 2016, Santa Barbara, CA, USA, June 13–17, 2016, pp 1–15

  19. Kocher P, Genkin D, Gruss D, Haas W, Hamburg M, Lipp M, Mangard S, Prescher T, Schwarz M, Yarom Y (2018) Spectre attacks: exploiting speculative execution. meltdownattack.com

  20. Moy Y, Wallenburg A (2010) Tokeneer: beyond formal program verification. In: Embedded real time software and systems, vol 24

  21. Kupferman O (2006) Sanity checks in formal verification. In: CONCUR 2006—concurrency theory, 17th international conference, CONCUR 2006, Bonn, Germany, August 27–30, 2006, Proceedings. Lecture notes in computer science, vol 4137, pp 37–51

  22. Serebryany K, Bruening D, Potapenko A, Vyukov D (2012) Addresssanitizer: a fast address sanity checker. In: Heiser G, Hsieh WC (eds) 2012 USENIX annual technical conference, Boston, MA, USA, June 13–15, 2012, pp 309–318. https://www.usenix.org/conference/atc12/technical-sessions/presentation/serebryany

  23. Kim Y, Kim M (2014) SAT-based bounded software model checking for embedded software: a case study. In: 21st Asia-Pacific Software Engineering Conference, APSEC 2014, Jeju, South Korea, December 1–4, 2014. Volume 1: research papers, pp 55–62

  24. Cook B, Khazem K, Kroening D, Tasiran S, Tautschnig M, Tuttle MR (2018) Model checking boot code from AWS data centers. In: Computer aided verification—30th international conference, CAV 2018, held as part of the federated logic conference, FloC 2018, Oxford, UK, July 14–17, 2018, Proceedings, Part II. Lecture notes in computer science, vol 10982, pp 467–486. https://OPTdoi.org/10.1007/978-3-319-96142-2_28

  25. Chudnov A, Collins N, Cook B, Dodds J, Huffman B, MacCárthaigh C, Magill S, Mertens E, Mullen E, Tasiran S, Tomb A, Westbrook E (2018) Continuous formal verification of Amazon s2n. In: Computer aided verification—30th international conference, CAV 2018, held as part of the federated logic conference, FloC 2018, Oxford, UK, July 14–17, 2018, Proceedings, Part II. Lecture notes in computer science, vol 10982, pp 430–446

  26. Cook B, Döbel B, Kroening D, Manthey N, Pohlack M, Polgreen E, Tautschnig M, Wieczorkiewicz P (2020) Using model checking tools to triage the severity of security bugs in the Xen hypervisor. In: 2020 formal methods in computer aided design, FMCAD 2020, Haifa, Israel, September 21–24, 2020, pp 185–193. https://OPTdoi.org/10.34727/2020/isbn.978-3-85448-042-6_26

  27. Fähndrich M, Barnett M, Logozzo F.: Embedded contract languages. In: Shin SY, Ossowski S, Schumacher M, Palakal MJ, Hung C (eds) Proceedings of the 2010 ACM Symposium on Applied Computing (SAC), Sierre, Switzerland, March 22–26, 2010, pp 2103–2110. https://doi.org/10.1145/1774088.1774531

Download references

Author information

Authors and Affiliations

  1. Department of Electrical and Computer Engineering, University of Waterloo, 200 University Ave. West, Waterloo, ON, N2L 3G1, Canada

    Siddharth Priya, Xiang Zhou, Yusen Su, Yuyan Bao & Arie Gurfinkel

  2. Department of Computer Science, The Technion, 3200003, Haifa, Israel

    Yakir Vizel

Authors
  1. Siddharth Priya
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xiang Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yusen Su
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yakir Vizel
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Yuyan Bao
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Arie Gurfinkel
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Siddharth Priya.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Priya, S., Zhou, X., Su, Y. et al. Verifying verified code. Innovations Syst Softw Eng 18, 335–346 (2022). https://doi.org/10.1007/s11334-022-00443-9

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11334-022-00443-9

Keywords

Mathematics Subject Classification

Search

Navigation