Skip to main content
SpringerLink
Log in

Benchmarking Open-Source Static Analyzers for Security Testing for C

  • Conference paper
  • First Online:
Leveraging Applications of Formal Methods, Verification and Validation: Tools and Trends (ISoLA 2020)

Part of the book series: Lecture Notes in Computer Science ((LNTCS,volume 12479))

Included in the following conference series:

Abstract

As the number of available static analysis security testing (SAST) tools grows, the more difficult it becomes for developers to decide which tool(s) to use. We report on our evaluation of 11 open-source general-purpose SAST tools for the C programming language on the SARD Juliet Test Suite and of six tools on the Wireshark software. In line with the previous work, we find that there is no single superior tool, though sound tools performed the best on the Juliet test cases.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    cf. Frama-C website (https://www.frama-c.com).

  2. 2.

    https://cwe.mitre.org/.

  3. 3.

    File CWE195_Signed_to_Unsigned_Conversion_Error__negative_malloc_18.c.

  4. 4.

    https://github.com/sosy-lab/sv-benchmarks.

  5. 5.

    https://www.debian.org.

  6. 6.

    https://cve.mitre.org.

  7. 7.

    https://nvd.nist.org.

  8. 8.

    https://cwe.mitre.org/data/definitions/888.html.

  9. 9.

    https://github.com/RohanKrishnamurthy/sastevaluation.

References

  1. Arusoaie, A., Ciobâcă, Ş., Craciun, V., Gavrilut, D., Lucanu, D.: A comparison of open-source static analysis tools for vulnerability detection in C/C++ code. In: SYNASC, pp. 161–168. IEEE Computer Society (2017)

    Google Scholar 

  2. Beyer, D.: Advances in automatic software verification: SV-COMP 2020. TACAS 2020. LNCS, vol. 12079, pp. 347–367. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-45237-7_21

    Chapter  Google Scholar 

  3. Calcagno, C., et al. : Moving fast with software verification. In: Havelund, K., Holzmann, G., Joshi, R. (eds.) NFM 2015. LNCS, vol. 9058, pp. 3–11. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-17524-9_1

  4. Chatzieleftheriou, G., Katsaros, P.: Test-driving static analysis tools in search of C code vulnerabilities. In: COMPSAC Workshops 2011, pp. 96–103. IEEE Computer Society (2011)

    Google Scholar 

  5. Delaitre, A., Stivalet, B., Black, P.E., Okun, V., Ribeiro, A., Cohen, T.S.: Sate V report: ten years of static analysis tool expositions. Tech. Rep. NIST-SP-500-326, NIST (2018). https://doi.org/10.6028/NIST.SP.500-326

  6. Goseva-Popstojanova, K., Perhinschi, A.: On the capability of static code analysis to detect security vulnerabilities. Inf. Softw. Technol. 68, 18–33 (2015)

    Article  Google Scholar 

  7. Herter, J., Kästner, D., Mallon, C., Wilhelm, R.: Benchmarking static code analyzers. Reliab. Eng. Syst. Saf. 188, 336–346 (2019)

    Article  Google Scholar 

  8. Kuhn, R., Raunak, M.S., Kacker, R.: Can reducing faults prevent vulnerabilities? IEEE Comput. 51(7), 82–85 (2018)

    Article  Google Scholar 

  9. Lu, B., Dong, W., Yin, L., Zhang, L.: Evaluating and integrating diverse bug finders for effective program analysis. In: Bu, L., Xiong, Y. (eds.) SATE 2018. LNCS, vol. 11293, pp. 51–67. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-04272-1_4

    Chapter  Google Scholar 

  10. Moerman, J., Smetsers, S., Schoolderman, M.: Evaluating the performance of open source static analysis tools. Bachelor thesis, Radboud University, The Netherlands, p. 24 (2018)

    Google Scholar 

  11. NAS-CAS: On analyzing static analysis tools. Technical report, National Security Agency Center for Assured Software (2017), https://media.blackhat.com/bh-us-11/ Willis/BH_US_11_WillisBritton_Analyzing_Static_Analysis_Tools_WP.pdf

  12. Prause, C., Gerlich, R., Gerlich, R.: Evaluating automated software verification tools. In: ICST 2018, pp. 343–353. IEEE Computer Society (2018)

    Google Scholar 

  13. Seacord, R.C.: The CERT\(\textregistered \) C Coding Standard, Second Edition: 98 Rules for Developing Safe, Reliable, and Secure Systems, 2nd edn. Addison-Wesley Professional (2014)

    Google Scholar 

  14. Shiraishi, S., Mohan, V., Marimuthu, H.: Test suites for benchmarks of static analysis tools. In: ISSRE Workshops 2015, pp. 12–15. IEEE Computer Society (2015)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. German Aerospace Center (DLR), Institute of Data Science, Jena, Germany

    Christoph Gentsch, Rohan Krishnamurthy & Thomas S. Heinze

Authors
  1. Christoph Gentsch
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Rohan Krishnamurthy
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Thomas S. Heinze
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Christoph Gentsch , Rohan Krishnamurthy or Thomas S. Heinze .

Editor information

Editors and Affiliations

  1. University of Limerick and Lero, Limerick, Ireland

    Tiziana Margaria

  2. TU Dortmund, Dortmund, Germany

    Bernhard Steffen

Rights and permissions

Reprints and permissions

Copyright information

© 2021 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Gentsch, C., Krishnamurthy, R., Heinze, T.S. (2021). Benchmarking Open-Source Static Analyzers for Security Testing for C. In: Margaria, T., Steffen, B. (eds) Leveraging Applications of Formal Methods, Verification and Validation: Tools and Trends. ISoLA 2020. Lecture Notes in Computer Science(), vol 12479. Springer, Cham. https://doi.org/10.1007/978-3-030-83723-5_13

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-83723-5_13

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-83722-8

  • Online ISBN: 978-3-030-83723-5

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation