research-article

Coverage-based Greybox Fuzzing as Markov Chain

Authors:

Van-Thuan Pham,

Abhik RoychoudhuryAuthors Info & Claims

CCS '16: Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security

Pages 1032 - 1043

https://doi.org/10.1145/2976749.2978428

Published: 24 October 2016 Publication History

Abstract

Coverage-based Greybox Fuzzing (CGF) is a random testing approach that requires no program analysis. A new test is generated by slightly mutating a seed input. If the test exercises a new and interesting path, it is added to the set of seeds; otherwise, it is discarded. We observe that most tests exercise the same few "high-frequency" paths and develop strategies to explore significantly more paths with the same number of tests by gravitating towards low-frequency paths. We explain the challenges and opportunities of CGF using a Markov chain model which specifies the probability that fuzzing the seed that exercises path i generates an input that exercises path j. Each state (i.e., seed) has an energy that specifies the number of inputs to be generated from that seed. We show that CGF is considerably more efficient if energy is inversely proportional to the density of the stationary distribution and increases monotonically every time that seed is chosen. Energy is controlled with a power schedule.

We implemented the exponential schedule by extending AFL. In 24 hours, AFLFAST exposes 3 previously unreported CVEs that are not exposed by AFL and exposes 6 previously unreported CVEs 7x faster than AFL. AFLFAST produces at least an order of magnitude more unique crashes than AFL.

References

[1]

F. Bellard. Qemu, a fast and portable dynamic translator. In Proceedings of the Annual Conference on USENIX Annual Technical Conference, ATEC '05, pages 41--41, 2005.

Digital Library

[2]

M. Böhme, B. C. d. S. Oliveira, and A. Roychoudhury. Regression tests to expose change interaction errors. In Proceedings of the 2013 9th Joint Meeting on Foundations of Software Engineering, ESEC/FSE 2013, pages 334--344, 2013.

Digital Library

[3]

M. Böhme and S. Paul. A probabilistic analysis of the efficiency of automated software testing. IEEE Transactions on Software Engineering, 42(4):345--360, April 2016.

Digital Library

[4]

S. Brin and L. Page. The anatomy of a large-scale hypertextual web search engine. In Proceedings of the Seventh International Conference on World Wide Web 7, WWW7, pages 107--117, 1998.

Digital Library

[5]

C. Cadar, D. Dunbar, and D. Engler. Klee: Unassisted and automatic generation of high-coverage tests for complex systems programs. In Proceedings of the 8th USENIX Conference on Operating Systems Design and Implementation, OSDI'08, pages 209--224, 2008.

Digital Library

[6]

S. K. Cha, M. Woo, and D. Brumley. Program-adaptive mutational fuzzing. In Proceedings of the 2015 IEEE Symposium on Security and Privacy, SP '15, pages 725--741, 2015.

Digital Library

[7]

Y. Chen, T. Su, C. Sun, Z. Su, and J. Zhao. Coverage-directed differential testing of jvm implementations. In PLDI'16, pages 85--99, 2016.

Digital Library

[8]

V. Chipounov, V. Kuznetsov, and G. Candea. S2e: A platform for in-vivo multi-path analysis of software systems. In ASPLOS XVI, pages 265--278, 2011.

Digital Library

[9]

V. Ganesh, T. Leek, and M. Rinard. Taint-based directed whitebox fuzzing. In Proceedings of the 31st International Conference on Software Engineering, ICSE '09, pages 474--484, 2009.

Digital Library

[10]

J. Geldenhuys, M. B. Dwyer, and W. Visser. Probabilistic symbolic execution. In Proceedings of the 2012 International Symposium on Software Testing and Analysis, ISSTA 2012, pages 166--176, 2012.

Digital Library

[11]

P. Godefroid, M. Y. Levin, and D. Molnar. Sage: Whitebox fuzzing for security testing. Queue, 10(1):20:20--20:27, Jan. 2012.

Digital Library

[12]

S. Kirkpatrick, C. Jr. Gelatt, and M. Vecchi. Optimization by simulated annealing. Science, 220(4598):671--680, 1983.

[13]

B. P. Miller, L. Fredriksen, and B. So. An empirical study of the reliability of unix utilities. Commun. ACM, 33(12):32--44, Dec. 1990.

Digital Library

[14]

J. R. Norris. Markov Chains (Cambridge Series in Statistical and Probabilistic Mathematics). Cambridge University Press, July 1998.

[15]

B. S. Pak. Hybrid fuzz testing: Discovering software bugs via fuzzing and symbolic execution. In Master's thesis, School of Computer Science, Carnegie Mellon University, 2012.

[16]

V.-T. Pham, M. Böhme, and A. Roychoudhury. Model-based whitebox fuzzing for program binaries. In Proceedings of the 31st IEEE/ACM International Conference on Automated Software Engineering, ASE, pages 552--562, 2016.

Digital Library

[17]

A. Rebert, S. K. Cha, T. Avgerinos, J. Foote, D. Warren, G. Grieco, and D. Brumley. Optimizing seed selection for fuzzing. In Proceedings of the 23rd USENIX Conference on Security Symposium, SEC'14, pages 861--875, 2014.

Digital Library

[18]

S. Sparks, S. Embleton, R. Cunningham, and C. Zou. Automated Vulnerability Analysis: Leveraging Control Flow for Evolutionary Input Crafting. In 23d Annual Computer Security Applications Conference (ACSAC), pages 477--486, 2007.

[19]

N. Stephens, J. Grosen, C. Salls, A. Dutcher, R. Wang, J. Corbetta, Y. Shoshitaishvili, C. Kruegel, and G. Vigna. Driller: Augmenting fuzzing through selective symbolic execution. In NDSS '16, pages 1--16, 2016.

[20]

Tool. Afl binary instrumentation. https://github.com/vrtadmin/moflow/tree/master/afl-dyninst. Accessed: 2016-05--13.

[21]

Tool. Afl vulnerability trophy case. http://lcamtuf.coredump.cx/afl/#bugs. Accessed: 2016-05--13.

[22]

Tool. American fuzzy lop (afl) fuzzer. http://lcamtuf.coredump.cx/afl/technical_details.txt. Accessed: 2016-05--13.

[23]

Tool. Peach Fuzzer Platform. http://www.peachfuzzer.com/products/peach-platform/. Accessed: 2016-05--13.

[24]

Tool. Pulling jpegs out of thin air. https://lcamtuf.blogspot.com/2014/11/pulling-jpegs-out-of-thin-air.html. Accessed: 2016-05--13.

[25]

Tool. SPIKE Fuzzer Platform. http://www.immunitysec.com. Accessed: 2016-05--13.

[26]

Tool. Suley Fuzzer. https://github.com/OpenRCE/sulley. Accessed: 2016-05--13.

[27]

Tool. Symbolic execution in vulnerability research. https://lcamtuf.blogspot.sg/2015/02/symbolic-execution-in-vuln-research.html. Accessed: 2016-05--13.

[28]

Tool. Zzuf: multi-purpose fuzzer. http://caca.zoy.org/wiki/zzuf. Accessed: 2016-05--13.

[29]

T. Wang, T. Wei, G. Gu, and W. Zou. Taintscope: A checksum-aware directed fuzzing tool for automatic software vulnerability detection. In Proceedings of the 2010 IEEE Symposium on Security and Privacy, SP '10, pages 497--512, 2010.

Digital Library

[30]

M. Woo, S. K. Cha, S. Gottlieb, and D. Brumley. Scheduling black-box mutational fuzzing. In Proceedings of the 2013 ACM SIGSAC Conference on Computer & Communications Security, CCS '13, pages 511--522, 2013.

Digital Library

Cited By

Lin FWang ZSasaki HDoerfert JGrosser TLeather HSadayappan P(2025)Teapot: Efficiently Uncovering Spectre Gadgets in COTS BinariesProceedings of the 23rd ACM/IEEE International Symposium on Code Generation and Optimization10.1145/3696443.3708936(553-569)Online publication date: 1-Mar-2025
https://dl.acm.org/doi/10.1145/3696443.3708936
Ye KZhu XXiao XWen SXue MXiang Y(2025)BazzAFL: Moving Fuzzing Campaigns Towards Bugs via Grouping Bug-Oriented SeedsIEEE Transactions on Dependable and Secure Computing10.1109/TDSC.2024.339179522:1(179-191)Online publication date: Jan-2025
https://doi.org/10.1109/TDSC.2024.3391795
Zhao JFu YWu YDong JHong R(2025)Thread-sensitive fuzzing for concurrency bug detectionComputers & Security10.1016/j.cose.2024.104171148(104171)Online publication date: Jan-2025
https://doi.org/10.1016/j.cose.2024.104171
Show More Cited By

Index Terms

Coverage-based Greybox Fuzzing as Markov Chain
1. Security and privacy
  1. Software and application security
  2. Systems security
    1. Vulnerability management
      1. Vulnerability scanners
2. Software and its engineering
  1. Software creation and management
    1. Software verification and validation
      1. Software defect analysis
        Software testing and debugging

Recommendations

Fine-grained Coverage-based Fuzzing
Fuzzing is a popular software testing method that discovers bugs by massively feeding target applications with automatically generated inputs. Many state-of-the-art fuzzers use branch coverage as a feedback metric to guide the fuzzing process. The fuzzer ...
Alphuzz: Monte Carlo Search on Seed-Mutation Tree for Coverage-Guided Fuzzing
ACSAC '22: Proceedings of the 38th Annual Computer Security Applications Conference

Coverage-based greybox fuzzing (CGF) has been approved to be effective in finding security vulnerabilities. Seed scheduling, the process of selecting an input as the seed from the seed pool for the next fuzzing iteration, plays a central role in CGF. ...
Beyond the Coverage Plateau: A Comprehensive Study of Fuzz Blockers (Registered Report)
FUZZING 2023: Proceedings of the 2nd International Fuzzing Workshop

Fuzzing and particularly code coverage-guided greybox fuzzing is highly successful in automated vulnerability discovery, as evidenced by the multitude of vulnerabilities uncovered in real-world software systems. However, results on large benchmarks ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

CCS '16: Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security

October 2016

1924 pages

ISBN:9781450341394

DOI:10.1145/2976749

General Chairs:
Edgar Weippl
SBA Research, Austria
,
Stefan Katzenbeisser
TU Darmstadt, CYSEC, Germany
,
Program Chairs:
Christopher Kruegel
University of California, Santa Barbara, USA
,
Andrew Myers
Cornell University, USA
,
Shai Halevi
IBM Research, USA

Copyright © 2016 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

SIGSAC: ACM Special Interest Group on Security, Audit, and Control

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 24 October 2016

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

National Research Foundation, Prime Minister's Office, Singapore

Conference

CCS'16

Sponsor:

SIGSAC

CCS'16: 2016 ACM SIGSAC Conference on Computer and Communications Security

October 24 - 28, 2016

Vienna, Austria

Acceptance Rates

CCS '16 Paper Acceptance Rate 137 of 831 submissions, 16%;

Overall Acceptance Rate 1,261 of 6,999 submissions, 18%

Upcoming Conference

CCS '25

Sponsor:
sigsac

ACM SIGSAC Conference on Computer and Communications Security

October 13 - 17, 2025

Taipei , Taiwan

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

368
Total Citations
View Citations
2,912
Total Downloads

Downloads (Last 12 months)507
Downloads (Last 6 weeks)69

Reflects downloads up to 03 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Lin FWang ZSasaki HDoerfert JGrosser TLeather HSadayappan P(2025)Teapot: Efficiently Uncovering Spectre Gadgets in COTS BinariesProceedings of the 23rd ACM/IEEE International Symposium on Code Generation and Optimization10.1145/3696443.3708936(553-569)Online publication date: 1-Mar-2025
https://dl.acm.org/doi/10.1145/3696443.3708936
Ye KZhu XXiao XWen SXue MXiang Y(2025)BazzAFL: Moving Fuzzing Campaigns Towards Bugs via Grouping Bug-Oriented SeedsIEEE Transactions on Dependable and Secure Computing10.1109/TDSC.2024.339179522:1(179-191)Online publication date: Jan-2025
https://doi.org/10.1109/TDSC.2024.3391795
Zhao JFu YWu YDong JHong R(2025)Thread-sensitive fuzzing for concurrency bug detectionComputers & Security10.1016/j.cose.2024.104171148(104171)Online publication date: Jan-2025
https://doi.org/10.1016/j.cose.2024.104171
Gao YZeng WLiu SZeng Y(2025)AutoFuzz: automatic fuzzer-sanitizer scheduling with multi-armed banditSoftware Quality Journal10.1007/s11219-025-09707-633:1Online publication date: 13-Jan-2025
https://doi.org/10.1007/s11219-025-09707-6
Wang MLiang JZhou CWu ZFu JSu ZLiao QGu BWu BJiang YBalzarotti DXu W(2024)Data coverage for guided fuzzingProceedings of the 33rd USENIX Conference on Security Symposium10.5555/3698900.3699041(2511-2526)Online publication date: 14-Aug-2024
https://dl.acm.org/doi/10.5555/3698900.3699041
Xiang YZhang XLiu PJi SXiao XLiang HXu JWang WBalzarotti DXu W(2024)Critical code guided directed greybox fuzzing for commitsProceedings of the 33rd USENIX Conference on Security Symposium10.5555/3698900.3699038(2459-2474)Online publication date: 14-Aug-2024
https://dl.acm.org/doi/10.5555/3698900.3699038
Feng SWu YXue WPan SZou DLiu YJin HBalzarotti DXu W(2024)FIREProceedings of the 33rd USENIX Conference on Security Symposium10.5555/3698900.3699005(1867-1884)Online publication date: 14-Aug-2024
https://dl.acm.org/doi/10.5555/3698900.3699005
Wang JKang YWu CHu YSun YRen JLai YXie MZhang CLi TWang ZBalzarotti DXu W(2024)OptFuzzProceedings of the 33rd USENIX Conference on Security Symposium10.5555/3698900.3698949(865-882)Online publication date: 14-Aug-2024
https://dl.acm.org/doi/10.5555/3698900.3698949
Singha RMondal RBeckett RKakarla SMillstein TVarghese GVanbever LZhang I(2024)MESSIProceedings of the 21st USENIX Symposium on Networked Systems Design and Implementation10.5555/3691825.3691881(1009-1023)Online publication date: 16-Apr-2024
https://dl.acm.org/doi/10.5555/3691825.3691881
Groppe JGroppe SSenf DMöller R(2024)There Are Infinite Ways to Formulate Code: How to Mitigate the Resulting Problems for Better Software Vulnerability DetectionInformation10.3390/info1504021615:4(216)Online publication date: 11-Apr-2024
https://doi.org/10.3390/info15040216
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten