research-article

GraphFuzz: library API fuzzing with lifetime-aware dataflow graphs

Authors:

Harrison Green,

Thanassis AvgerinosAuthors Info & Claims

ICSE '22: Proceedings of the 44th International Conference on Software Engineering

Pages 1070 - 1081

https://doi.org/10.1145/3510003.3510228

Published: 05 July 2022 Publication History

Abstract

We present the design and implementation of GraphFuzz, a new structure-, coverage- and object lifetime-aware fuzzer capable of automatically testing low-level Library APIs. Unlike other fuzzers, GraphFuzz models sequences of executed functions as a dataflow graph, thus enabling it to perform graph-based mutations both at the data and at the execution trace level. GraphFuzz comes with an automated specification generator to minimize the developer integration effort.

We use GraphFuzz to analyze Skia---the rigorously tested Google Chrome graphics library---and benchmark GraphFuzz-generated fuzzing harnesses against hand-optimized, painstakingly written libFuzzer harnesses. We find that GraphFuzz generates test cases that achieve 2--3x more code coverage on average with minimal development effort, and also uncovered previous unknown defects in the process. We demonstrate GraphFuzz's applicability on low-level APIs by analyzing four additional open-source libraries and finding dozens of previously unknown defects. All security relevant findings have already been reported and fixed by the developers.

Last, we open-source GraphFuzz under a permissive license and provide code to reproduce all results in this paper.

References

[1]

K. Serebryany, "Oss-fuzz-google's continuous fuzzing service for open source software," 2017.

[2]

"Fuzzing for safety critical systems." https://forallsecure.com/safety-critical. Accessed: 2021-09-03.

[3]

K. Serebryany, "libfuzzer-a library for coverage-guided fuzz testing," LLVM project, 2015.

[4]

C. Lattner and V. Adve, "Llvm: A compilation framework for lifelong program analysis & transformation," in International Symposium on Code Generation and Optimization, 2004. CGO 2004., pp. 75--86, IEEE, 2004.

Digital Library

[5]

"google/libprotobuf-mutator," June 2021. original-date: 2017-01-11T22:57:02Z.

[6]

X. Yang, Y. Chen, E. Eide, and J. Regehr, "Finding and understanding bugs in c compilers," in Proceedings of the 32nd ACM SIGPLAN conference on Programming language design and implementation, pp. 283--294, 2011.

[7]

D. Babić, S. Bucur, Y. Chen, F. Ivančić, T. King, M. Kusano, C. Lemieux, L. Szekeres, and W. Wang, "Fudge: fuzz driver generation at scale," in Proceedings of the 2019 27th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering, pp. 975--985, 2019.

[8]

K. Serebryany, D. Bruening, A. Potapenko, and D. Vyukov, "Addresssanitizer: A fast address sanity checker," in 2012 {USENIX} Annual Technical Conference ({USENIX} {ATC} 12), pp. 309--318, 2012.

[9]

J. Ruderman, "Introducing jsfunfuzz," URL http://www.squarefree.com/2007/08/02/introducing-jsfunfuzz, vol. 20, pp. 25--29, 2007.

[10]

C. Aschermann, T. Frassetto, T. Holz, P. Jauernig, A.-R. Sadeghi, and D. Teuchert, "Nautilus: Fishing for deep bugs with grammars." in NDSS, 2019.

[11]

V. Atlidakis, R. Geambasu, P. Godefroid, M. Polishchuk, and B. Ray, "Pythia: grammar-based fuzzing of rest apis with coverage-guided feedback and learning-based mutations" arXiv preprint arXiv:2005.11498, 2020.

[12]

"chromium/src.git - Git at Google."

[13]

H. Han, D. Oh, and S. K. Cha, "Codealchemist: Semantics-aware code generation to find vulnerabilities in javascript engines.," in NDSS, 2019.

[14]

H. Han and S. K. Cha, "Imf: Inferred model-based fuzzer" in Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security, pp. 2345--2358, 2017.

Digital Library

[15]

K. Ispoglou, D. Austin, V. Mohan, and M. Payer, "Fuzzgen: Automatic fuzzer generation" in 29th {USENIX} Security Symposium ({USENIX} Security 20), pp. 2271--2287, 2020.

[16]

D. Vyukov, "Syzkaller," 2015.

[17]

V. J. M. Manes, H. Han, C. Han, S. K. Cha, M. Egele, E. J. Schwartz, and M. Woo, "The Art, Science, and Engineering of Fuzzing: A Survey" IEEE Transactions on Software Engineering, pp. 1--1, 2019.

[18]

C. Holler, K. Herzig, and A. Zeller, "Fuzzing with code fragments" in 21st {USENIX} Security Symposium ({USENIX} Security 12), pp. 445--458, 2012.

[19]

S. Veggalam, S. Rawat, I. Haller, and H. Bos, "Ifuzzer: An evolutionary interpreter fuzzer using genetic programming" in European Symposium on Research in Computer Security, pp. 581--601, Springer, 2016.

[20]

V. Atlidakis, P. Godefroid, and M. Polishchuk, "Restler: Stateful rest api fuzzing" in 2019 IEEE/ACM 41st International Conference on Software Engineering (ICSE), pp. 748--758, IEEE, 2019.

[21]

"MozillaSecurity/dharma," May 2021. original-date: 2015-03-25T17:56:23Z.

[22]

J. Jiang, H. Xu, and Y. Zhou, "Rulf: Rust library fuzzing via api dependency graph traversal," arXiv preprint arXiv:2104.12064, 2021.

[23]

"Skia: The 2d graphics library." https://skia.org/. Accessed: 2021-09-03.

[24]

"Rdkit: Open-source cheminformatics" http://www.rdkit.org. Accessed: 2021-09-03.

[25]

"Sqlite: In-memory database" https://www.sqlite.org/. Accessed: 2021-09-03.

[26]

G. Guennebaud, B. Jacob, et al., "Eigen v3." http://eigen.tuxfamily.org, 2010.

[27]

"Iowow: C11 key/value database engine." https://iowow.io/. Accessed: 2021-09-03.

Cited By

Mancini FDa Canal SBianchi G(2024)AMFuzz: Black-Box Fuzzing of 5G Core Networks2024 19th Wireless On-Demand Network Systems and Services Conference (WONS)10.23919/WONS60642.2024.10449510(17-24)Online publication date: 29-Jan-2024
https://doi.org/10.23919/WONS60642.2024.10449510
Takashima YCho CMartins RJia LPăsăreanu C(2024)Crabtree: Rust API Test Synthesis Guided by Coverage and TypeProceedings of the ACM on Programming Languages10.1145/36897338:OOPSLA2(618-647)Online publication date: 8-Oct-2024
https://dl.acm.org/doi/10.1145/3689733
Sun Y(2024)Automated Generation and Compilation of Fuzz Driver Based on Large Language ModelsProceedings of the 2024 9th International Conference on Cyber Security and Information Engineering10.1145/3689236.3689272(461-468)Online publication date: 15-Sep-2024
https://dl.acm.org/doi/10.1145/3689236.3689272
Show More Cited By

Recommendations

A Static Approach to Prioritizing JUnit Test Cases

Test case prioritization is used in regression testing to schedule the execution order of test cases so as to expose faults earlier in testing. Over the past few years, many test case prioritization techniques have been proposed in the literature. Most ...
Using coverage criteria on RepOK to reduce bounded-exhaustive test suites
TAP'12: Proceedings of the 6th international conference on Tests and Proofs

Bounded-exhaustive exploration of test case candidates is a commonly employed approach for test generation in some contexts. Even when small bounds are used for test generation, executing the obtained tests may become prohibitive, despite the time for ...
Spartan: a spectral and entropy-based partial-scan and test point insertion algorithm

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

ICSE '22: Proceedings of the 44th International Conference on Software Engineering

May 2022

2508 pages

ISBN:9781450392211

DOI:10.1145/3510003

General Chair:
Matthew B Dwyer
University of Virginia
,
Program Chairs:
Daniela Damian
University of Victoria, Canada
,
Andreas Zeller
CISPA, Germany

Copyright © 2022 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGSOFT: ACM Special Interest Group on Software Engineering

In-Cooperation

IEEE CS

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 05 July 2022

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Qualifiers

Research-article

Conference

ICSE '22

Sponsor:

SIGSOFT

ICSE '22: 44th International Conference on Software Engineering

May 21 - 29, 2022

Pennsylvania, Pittsburgh

Acceptance Rates

Overall Acceptance Rate 276 of 1,856 submissions, 15%

Upcoming Conference

ICSE 2025

2025 IEEE/ACM 46th International Conference on Software Engineering

April 26 - May 3, 2025

Ottawa , ON , Canada

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

15
Total Citations
View Citations
450
Total Downloads

Downloads (Last 12 months)189
Downloads (Last 6 weeks)16

Reflects downloads up to 02 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Mancini FDa Canal SBianchi G(2024)AMFuzz: Black-Box Fuzzing of 5G Core Networks2024 19th Wireless On-Demand Network Systems and Services Conference (WONS)10.23919/WONS60642.2024.10449510(17-24)Online publication date: 29-Jan-2024
https://doi.org/10.23919/WONS60642.2024.10449510
Takashima YCho CMartins RJia LPăsăreanu C(2024)Crabtree: Rust API Test Synthesis Guided by Coverage and TypeProceedings of the ACM on Programming Languages10.1145/36897338:OOPSLA2(618-647)Online publication date: 8-Oct-2024
https://dl.acm.org/doi/10.1145/3689733
Sun Y(2024)Automated Generation and Compilation of Fuzz Driver Based on Large Language ModelsProceedings of the 2024 9th International Conference on Cyber Security and Information Engineering10.1145/3689236.3689272(461-468)Online publication date: 15-Sep-2024
https://dl.acm.org/doi/10.1145/3689236.3689272
Li AHuang MLemieux CPadhye RBöhme MNoller YSzekeres L(2024)The Havoc Paradox in Generator-Based Fuzzing (Registered Report)Proceedings of the 3rd ACM International Fuzzing Workshop10.1145/3678722.3685529(3-12)Online publication date: 13-Sep-2024
https://dl.acm.org/doi/10.1145/3678722.3685529
Lyu YXie YChen PChen HLuo BLiao XXu JKirda ELie D(2024)Prompt Fuzzing for Fuzz Driver GenerationProceedings of the 2024 on ACM SIGSAC Conference on Computer and Communications Security10.1145/3658644.3670396(3793-3807)Online publication date: 2-Dec-2024
https://dl.acm.org/doi/10.1145/3658644.3670396
Suo CChen JLiu SJiang JZhao YWang JChristakis MPradel M(2024)Fuzzing MLIR Compiler Infrastructure via Operation Dependency AnalysisProceedings of the 33rd ACM SIGSOFT International Symposium on Software Testing and Analysis10.1145/3650212.3680360(1287-1299)Online publication date: 11-Sep-2024
https://dl.acm.org/doi/10.1145/3650212.3680360
Zhu XZhang SLi CWen SXiang Y(2024)Fuzzing Android Native System Libraries via Dynamic Data Dependency GraphIEEE Transactions on Information Forensics and Security10.1109/TIFS.2024.336947919(3733-3744)Online publication date: 23-Feb-2024
https://dl.acm.org/doi/10.1109/TIFS.2024.3369479
Schloegel MBars NSchiller NBernhard LScharnowski TCrump AAle-Ebrahim ABissantz NMuench MHolz T(2024)SoK: Prudent Evaluation Practices for Fuzzing2024 IEEE Symposium on Security and Privacy (SP)10.1109/SP54263.2024.00137(1974-1993)Online publication date: 19-May-2024
https://doi.org/10.1109/SP54263.2024.00137
Liu YWang YJia XZhang ZSu P(2024)AFGen: Whole-Function Fuzzing for Applications and Libraries2024 IEEE Symposium on Security and Privacy (SP)10.1109/SP54263.2024.00011(1901-1919)Online publication date: 19-May-2024
https://doi.org/10.1109/SP54263.2024.00011
Nourry OKashiwa YLin BBavota GLanza MKamei Y(2023)The Human Side of Fuzzing: Challenges Faced by Developers during Fuzzing ActivitiesACM Transactions on Software Engineering and Methodology10.1145/361166833:1(1-26)Online publication date: 23-Nov-2023
https://dl.acm.org/doi/10.1145/3611668
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten