Skip to main content
SpringerLink
Log in
SpringerLink
Log in

FuSeBMC: An Energy-Efficient Test Generator for Finding Security Vulnerabilities in C Programs

  • Conference paper
  • First Online:
Tests and Proofs (TAP 2021)

Part of the book series: Lecture Notes in Computer Science ((LNPSE,volume 12740))

Included in the following conference series:

Abstract

We describe and evaluate a novel approach to automated test generation that exploits fuzzing and Bounded Model Checking (BMC) engines to detect security vulnerabilities in C programs. We implement this approach in a new tool FuSeBMC that explores and analyzes the target C program by injecting labels that guide the engines to produce test cases. FuSeBMC also exploits a selective fuzzer to produce test cases for the labels that fuzzing and BMC engines could not produce test cases. Lastly, we manage each engine’s execution time to improve FuSeBMC’s energy consumption. We evaluate FuSeBMC by analysing the results of its participation in Test-Comp 2021 whose two main categories evaluate a tool’s ability to provide code coverage and bug detection. The competition results show that FuSeBMC performs well compared to the state-of-the-art software testing tools. FuSeBMC achieved 3 awards in the Test-Comp 2021: first place in the Cover-Error category, second place in the Overall category, and third place in the Low Energy Consumption category.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    https://gitlab.com/sosy-lab/software/test-format/.

  2. 2.

    https://github.com/kaled-alshmrany/FuSeBMC.

  3. 3.

    https://doi.org/10.5281/zenodo.4710599.

References

  1. Rodriguez, M., Piattini, M., Ebert, C.: Software verification and validation technologies and tools. IEEE Softw. 36(2), 13–24 (2019)

    Article  Google Scholar 

  2. Airbus issues software bug alert after fatal plane crash. The Guardian, May 2015. https://tinyurl.com/xw67wtd9. Accessed Mar 2021

  3. Liu, B., Shi, L., Cai, Z., Li, M.: Software vulnerability discovery techniques: a survey. In: 2012 Fourth International Conference on Multimedia Information Networking and Security, pp. 152–156. IEEE (2012)

    Google Scholar 

  4. Clarke, E.M., Emerson, E.A.: Design and synthesis of synchronization skeletons using branching time temporal logic. In: Grumberg, O., Veith, H. (eds.) 25 Years of Model Checking, pp. 196–215 (2008)

    Google Scholar 

  5. Godefroid, P.: Fuzzing: hack, art, and science. Commun. ACM 63(2), 70–76 (2020)

    Article  Google Scholar 

  6. Clarke, E.M., Klieber, W., Nováček, M., Zuliani, P.: Model checking and the state explosion problem. In: Meyer, B., Nordio, M. (eds.) LASER 2011. LNCS, vol. 7682, pp. 1–30. Springer, Heidelberg (2012). https://doi.org/10.1007/978-3-642-35746-6_1

    Chapter  Google Scholar 

  7. Shameng, W., Feng Chao, E.A.: Testing network protocol binary software with selective symbolic execution. In: CIS, pp. 318–322. IEEE (2016)

    Google Scholar 

  8. Beyer, D.: 3rd competition on software testing (test-comp 2021) (2021)

    Google Scholar 

  9. Miller, B.P., et al.: Fuzz revisited: a re-examination of the reliability of UNIX utilities and services. Technical report, UW-Madison (1995)

    Google Scholar 

  10. King, J.C.: Symbolic execution and program testing. Commun. ACM 19(7), 385–394 (1976)

    Article  MathSciNet  Google Scholar 

  11. Faria, J.: Inspections, revisions and other techniques of software static analysis. Software Testing and Quality, Lecture, vol. 9 (2008)

    Google Scholar 

  12. Qin, S., Kim, H.S.: LIFT: a low-overhead practical information flow tracking system for detecting security attacks. In: MICRO 2006, pp. 135–148. IEEE (2006)

    Google Scholar 

  13. Ognawala, S., Kilger, F., Pretschner, A.: Compositional fuzzing aided by targeted symbolic execution. arXiv preprint arXiv:1903.02981 (2019)

  14. Basak Chowdhury, A., Medicherla, R.K., Venkatesh, R.: VeriFuzz: program aware fuzzing. In: Beyer, D., Huisman, M., Kordon, F., Steffen, B. (eds.) TACAS 2019. LNCS, vol. 11429, pp. 244–249. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-17502-3_22

    Chapter  Google Scholar 

  15. Le, H.M.: LLVM-based hybrid fuzzing with LibKluzzer (competition contribution). In: FASE, pp. 535–539 (2020)

    Google Scholar 

  16. Biere, A.: Bounded model checking. In: Biere, A., Heule, M., van Maaren, H., Walsh, T. (eds.) Handbook of Satisfiability. Frontiers in Artificial Intelligence and Applications, vol. 185, pp. 457–481. IOS Press (2009)

    Google Scholar 

  17. Cordeiro, L.C., Fischer, B., Marques-Silva, J.: SMT-based bounded model checking for embedded ANSI-C software. IEEE Trans. Software Eng. 38(4), 957–974 (2012)

    Article  Google Scholar 

  18. Beyer, D.: Second competition on software testing: Test-Comp 2020. In: Wehrheim, H., Cabot, J. (eds.) Fundamental Approaches to Software Engineering. FASE 2020. LNCS, vol. 12076, pp. 505–519. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-45234-6_25

  19. Cadar, C., Dunbar, D., Engler, D.R.: KLEE: unassisted and automatic generation of high-coverage tests for complex systems programs. In: OSDI, vol. 8, pp. 209–224 (2008)

    Google Scholar 

  20. Beyer, D., Keremoglu, M.E.: CPAchecker: a tool for configurable software verification. In: Gopalakrishnan, G., Qadeer, S. (eds.) CAV 2011. LNCS, vol. 6806, pp. 184–190. Springer, Heidelberg (2011). https://doi.org/10.1007/978-3-642-22110-1_16

    Chapter  Google Scholar 

  21. Alshmrany, K.M., Menezes, R.S., Gadelha, M.R., Cordeiro, L.C.: FuSeBMC: a white-box fuzzer for finding security vulnerabilities in c programs. In: 24th International Conference on Fundamental Approaches to Software Engineering (FASE), vol. 12649, pp. 363–367 (2020)

    Google Scholar 

  22. Munea, T.L., Lim, H., Shon, T.: Network protocol fuzz testing for information systems and applications: a survey and taxonomy. Multimedia Tools Appl. 75(22), 14745–14757 (2016)

    Article  Google Scholar 

  23. Wang, J., Guo, T., Zhang, P., Xiao, Q.: A model-based behavioral fuzzing approach for network service. In: 2013 Third International Conference on IMCCC, pp. 1129–1134. IEEE (2013)

    Google Scholar 

  24. Baldoni, R., Coppa, E., D’elia, D.C., Demetrescu, C., Finocchi, I.: A survey of symbolic execution techniques. ACM Comput. Surv. 51, 1–39 (2018)

    Google Scholar 

  25. Chipounov, V., Georgescu, V., Zamfir, C., Candea, G.: Selective symbolic execution. In: Proceedings of the 5th Workshop on HotDep (2009)

    Google Scholar 

  26. Black, P.E., Bojanova, I.: Defeating buffer overflow: a trivial but dangerous bug. IT Prof. 18(6), 58–61 (2016)

    Article  Google Scholar 

  27. Zhang, S., Zhu, J., Liu, A., Wang, W., Guo, C., Xu, J.: A novel memory leak classification for evaluating the applicability of static analysis tools. In: 2018 IEEE International Conference on Progress in Informatics and Computing (PIC), pp. 351–356. IEEE (2018)

    Google Scholar 

  28. Jimenez, W., Mammar, A., Cavalli, A.: Software vulnerabilities, prevention and detection methods: a review. In: Security in Model-Driven Architecture, vol. 215995, p. 215995 (2009)

    Google Scholar 

  29. Boudjema, E.H., Faure, C., Sassolas, M., Mokdad, L.: Detection of security vulnerabilities in C language applications. Secur. Priv. 1(1), e8 (2018)

    Article  Google Scholar 

  30. US-CERT: Understanding denial-of-service attacks \(|\) CISA (2009)

    Google Scholar 

  31. Cisco: Cisco IOS XE software cisco discovery protocol memory leak vulnerability (2018)

    Google Scholar 

  32. Clang documentation (2015). http://clang.llvm.org/docs/index.html. Accessed Aug 2019

  33. Rocha, H.O., Barreto, R.S., Cordeiro, L.C.: Hunting memory bugs in C programs with Map2Check. In: Chechik, M., Raskin, J.-F. (eds.) TACAS 2016. LNCS, vol. 9636, pp. 934–937. Springer, Heidelberg (2016). https://doi.org/10.1007/978-3-662-49674-9_64

    Chapter  Google Scholar 

  34. Rocha, H., Menezes, R., Cordeiro, L.C., Barreto, R.: Map2Check: using symbolic execution and fuzzing. In: Biere, A., Parker, D. (eds.) Tools and Algorithms for the Construction and Analysis of Systems. TACAS 2020. LNCS, vol. 12079, pp. 403–407. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-45237-7_29

  35. Gadelha, M.R., Monteiro, F., Cordeiro, L., Nicole, D.: ESBMC v6.0: verifying C programs using k-induction and invariant inference. In: Beyer, D., Huisman, M., Kordon, F., Steffen, B. (eds.) TACAS 2019. LNCS, vol. 11429, pp. 209–213. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-17502-3_15

    Chapter  Google Scholar 

  36. Gadelha, M.R., Monteiro, F.R., Morse, J., Cordeiro, L.C., Fischer, B., Nicole, D.A.: ESBMC 5.0: an industrial-strength C model checker. In: Proceedings of the 33rd ACM/IEEE International Conference on Automated Software Engineering, pp. 888–891 (2018)

    Google Scholar 

  37. Beyer, D., Lemberger, T.: TestCov: robust test-suite execution and coverage measurement. In: 2019 34th IEEE/ACM International Conference on Automated Software Engineering (ASE), pp. 1074–1077. IEEE (2019)

    Google Scholar 

  38. Lopes, B.C., Auler, R.: Getting started with LLVM core libraries. Packt Publishing Ltd. (2014)

    Google Scholar 

  39. Beyer, D.: Status report on software testing: Test-Comp 2021. In: Guerra, E., Stoelinga, M. (eds.) Fundamental Approaches to Software Engineering FASE 2021. LNCS, vol. 12649, pp. 341–357. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-71500-7_17

  40. Beyer, D.: Software verification: 10th comparative evaluation (SV-COMP 2021). In: Groote, J.F., Larsen, K.G. (eds.) Tools and Algorithms for the Construction and Analysis of Systems. TACAS 2021. LNCS, vol. 12652, pp. 401–422. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-72013-1_24

  41. Chalupa, M., Novák, J., Strejček, J.: Symbiotic  8: parallel and targeted test generation. FASE 2021. LNCS, vol. 12649, pp. 368–372. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-71500-7_20

    Chapter  Google Scholar 

  42. Beyer, D., Wendler, P.: CPU energy meter: a tool for energy-aware algorithms engineering. TACAS 2020. LNCS, vol. 12079, pp. 126–133. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-45237-7_8

    Chapter  Google Scholar 

  43. Barton, J.H., Czeck, E.W., Segall, Z.Z., Siewiorek, D.P.: Fault injection experiments using fiat. IEEE Trans. Comput. 39(4), 575–582 (1990)

    Article  Google Scholar 

  44. Böhme, M., Pham, V.-T., Nguyen, M.-D., Roychoudhury, A.: Directed greybox fuzzing. In: Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security, pp. 2329–2344 (2017)

    Google Scholar 

  45. American fuzzy lop (2021). https://lcamtuf.coredump.cx/afl/

  46. Serebryany, K.: libFuzzer-a library for coverage-guided fuzz testing. LLVM project (2015)

    Google Scholar 

  47. Gorbunov, S., Rosenbloom, A.: AutoFuzz: automated network protocol fuzzing framework. IJCSNS 10(8), 239 (2010)

    Google Scholar 

  48. Eddington, M.: Peach fuzzing platform. Peach Fuzzer, vol. 34 (2011)

    Google Scholar 

  49. Jaffar, J., Maghareh, R., Godboley, S., Ha, X.-L.: TracerX: dynamic symbolic execution with interpolation (competition contribution). In: FASE, pp. 530–534 (2020)

    Google Scholar 

  50. Song, J., Cadar, C., Pietzuch, P.: SymbexNet: testing network protocol implementations with symbolic execution and rule-based specifications. In: IEEE TSE, vol. 40, no. 7, pp. 695–709 (2014)

    Google Scholar 

  51. Sasnauskas, R., Kaiser, P., Jukić, R.L., Wehrle, K.: Integration testing of protocol implementations using symbolic distributed execution. In: ICNP, pp. 1–6. IEEE (2012)

    Google Scholar 

  52. Le, H.M.: LLVM-based hybrid fuzzing with LibKluzzer (competition contribution). In: Wehrheim, H., Cabot, J. (eds.) Fundamental Approaches to Software Engineering. FASE 2020. LNCS, vol. 12076, pp. 535–539. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-45234-6_29

  53. Stephens, N., et al.: Driller: augmenting fuzzing through selective symbolic execution. In: NDSS, pp. 1–16 (2016)

    Google Scholar 

  54. Pak, B.S.: Hybrid fuzz testing: discovering software bugs via fuzzing and symbolic execution. School of Computer Science Carnegie Mellon University (2012)

    Google Scholar 

  55. Noller, Y., Kersten, R., Păsăreanu, C.S.: Badger: complexity analysis with fuzzing and symbolic execution. In: Proceedings of the 27th ACM SIGSOFT International Symposium on Software Testing and Analysis, pp. 322–332 (2018)

    Google Scholar 

  56. Păsăreanu, C.S., Rungta, N.: Symbolic pathfinder: symbolic execution of Java bytecode. In: Proceedings of the IEEE/ACM International Conference on Automated Software Engineering, pp. 179–180 (2010)

    Google Scholar 

  57. Ognawala, S., Hutzelmann, T., Psallida, E., Pretschner, A.: Improving function coverage with munch: a hybrid fuzzing and directed symbolic execution approach. In: Proceedings of the 33rd Annual ACM Symposium on Applied Computing, pp. 1475–1482 (2018)

    Google Scholar 

  58. Godefroid, P., Levin, M.Y., Molnar, D.: SAGE: whitebox fuzzing for security testing. Queue 10(1), 20–27 (2012)

    Article  Google Scholar 

  59. Wang, M., et al.: SAFL: increasing and accelerating testing coverage with symbolic execution and guided fuzzing. In: Proceedings of the 40th International Conference on Software Engineering: Companion Proceedings, pp. 61–64 (2018)

    Google Scholar 

  60. He, J., Balunović, M., Ambroladze, N., Tsankov, P., Vechev, M.: Learning to fuzz from symbolic execution with application to smart contracts. In: Proceedings of the 2019 ACM SIGSAC Conference on Computer and Communications Security, pp. 531–548 (2019)

    Google Scholar 

  61. Cordeiro, L.C.: SMT-based bounded model checking for multi-threaded software in embedded systems. In: International Conference on Software Engineering, pp. 373–376. ACM (2010)

    Google Scholar 

  62. Pereira, P.A., et al.: Verifying CUDA programs using SMT-based context-bounded model checking. In: Ossowski, S. (ed.) Annual ACM Symposium on Applied Computing, pp. 1648–1653. ACM (2016)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. University of Manchester, Manchester, UK

    Kaled M. Alshmrany, Mohannad Aldughaim, Ahmed Bhayat & Lucas C. Cordeiro

  2. Institute of Public Administration, Jeddah, Saudi Arabia

    Kaled M. Alshmrany

Authors
  1. Kaled M. Alshmrany
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Mohannad Aldughaim
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ahmed Bhayat
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Lucas C. Cordeiro
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Kaled M. Alshmrany .

Editor information

Editors and Affiliations

  1. Université d’Orléans, Orléans, France

    Frédéric Loulergue

  2. Graz University of Technology, Graz, Austria

    Franz Wotawa

A Appendix

A Appendix

1.1 A.1 Artifact

We have set up a zenodo entry that contains the necessary materials to reproduce the results given in this paper: https://doi.org/10.5281/zenodo.4710599. Also, it contains instructions to run the tool.

1.2 A.2 Tool Availability

FuSeBMC contents are publicly available in our repository in GitHub under the terms of the MIT License. FuSeBMC provides, besides other files, a script called fusebmc.py. In order to run our fusebmc.py script, one must set the architecture (i.e., 32 or 64-bit), the competition strategy (i.e., k-induction, falsification, or incremental BMC), the property file path, and the benchmark path. FuSeBMC participated in the 3rd international competition, Test-Comp 21, and met all the requirements each tool needs to meet to qualify and participate. The results in our paper are also available on the Test-Comp 21 website. Finally, instructions for building FuSeBMC from the source code are given in the file README.md in our GitHub repository, including the description of all dependencies.

1.3 A.3 Tool Setup

FuSeBMC is available to download from the link.Footnote 3 To generate test cases for a C program a command of the following form is run:

figure h

where -a sets the architecture (either 32- or 64-bit), -p sets the property file path, -s sets the strategy (one of kinduction, falsi, incr, or fixed) and <file>.c is the C program to be checked. FuSeBMC produces the test cases in the XML format.

Rights and permissions

Reprints and permissions

Copyright information

© 2021 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Alshmrany, K.M., Aldughaim, M., Bhayat, A., Cordeiro, L.C. (2021). FuSeBMC: An Energy-Efficient Test Generator for Finding Security Vulnerabilities in C Programs. In: Loulergue, F., Wotawa, F. (eds) Tests and Proofs. TAP 2021. Lecture Notes in Computer Science(), vol 12740. Springer, Cham. https://doi.org/10.1007/978-3-030-79379-1_6

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-79379-1_6

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-79378-4

  • Online ISBN: 978-3-030-79379-1

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation