Abstract
Security assurance cases (security cases) are used to represent claims for evidence-based assurance of security properties in software. A security case uses evidence to argue that a particular claim is true, e.g., buffer overflows cannot happen. Evidence may be generated with a variety of methods. Random negative testing (fuzz testing) has become a popular method for creating evidence for the security of software. However, traditional fuzz testing is undirected and provides only weak evidence for specific assurance concerns, unless significant resources are allocated for extensive testing. This paper presents a method to apply fuzz testing in a targeted way to more economically support the creation of evidence for specific security assurance cases. Our experiments produced results with target code coverage comparable to an exhaustive fuzz test run while significantly reducing the test execution time when compared to exhaustive methods. These results provide specific evidence for security cases and provide improved assurance.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Lipner, S.: The trustworthy computing security development lifecycle. In: 20th IEEE Computer Security Applications Conference, pp. 2–13. IEEE (2004)
Agudo, I., Vivas, J., Lopez, J.: Security assurance during the software development cycle. In: International Conference on Computer Systems and Technologies and Workshop for PhD Students in Computing, p. 20. ACM (2009)
Kelly, T., Weaver, R.: The goal structuring notation-a safety argument notation. In: Dependable Systems and Networks 2004 Workshop on Assurance Cases. Citeseer (2004)
Godefroid, P., Levin, M., Molnar, D.: Automated whitebox fuzz testing. In: NDSS, vol. 8 (2008)
Takanen, A., Demott, J., Miller, C.: Fuzzing for software security testing and quality assurance. Artech House (2008)
Sutton, M., Greene, A., Amini, P.: Fuzzing: brute force vulnerability discovery. Addison-Wesley Professional (2007)
Miller, B.P., Fredriksen, L., So, B.: An empirical study of the reliability of UNIX utilities. Commun. ACM 33(12), 32–44 (1990)
DeMott, J.: The evolving art of fuzzing. Technical report, DEF CON, vol. 14 (2006)
Marshall, A., Howard, M., Bugher, G., et al.: Security best practices for developing windows azure applications. Technical report, Microsoft Corporation (2010)
Howard, M., Lipner, S.: The security development lifecycle, vol. 11. Microsoft Press (2009)
Goertzel, K.M., Winograd, T., McKinley, H.L., et al.: Software security assurance: a State-of-Art Report (SAR). DTIC Document (2007)
Wang, T., Wei, T., Gu, G., Zou, W.: TaintScope: A checksum-aware directed fuzzing tool for automatic software vulnerability detection. In: IEEE Symposium on Security and Privacy (SP), pp. 497–512. IEEE (2010)
Godefroid, P., Klarlund, N., Sen, K.: DART: directed automated random testing. ACM Sigplan Not. 40(6), 213–223 (2005)
Cadar, C., Dunbar, D., Engler, D.R.: KLEE: unassisted and automatic generation of high-coverage tests for complex systems programs. OSDI 8, 209–224 (2008)
Ganesh, V., Leek, T., Rinard, M.: Taint-based directed whitebox fuzzing. In: IEEE 31st International Conference on Software Engineering, pp. 474–484. IEEE (2009)
Wu, Z., Atwood, J.W., Zhu, X.: A new fuzzing technique for software vulnerability mining. In: IEEE CONSEG, vol. 9. IEEE (2009)
Jain, L.C., Karr, C.L.: Introduction to evolutionary computing techniques. In: Electronic Technology Directions, pp. 122–127 (1995)
Holland, J.H.: Adaptation in natural and artificial systems: an introductory analysis with applications to biology, control, and artificial intelligence. U. Michigan Press (1975)
Belew, R.K., McInerney, J., Schraudolph, N.N.: Evolving networks: using the genetic algorithm with connectionist learning. Citeseer (1990)
Whitley, D.: A genetic algorithm tutorial. Stat. Comput. 4(2), 65–85 (1994)
Eiben, A.E., Smith, J.E.: Introduction to Evolutionary Computing. Springer, Berlin (2010)
Chess, B., McGraw, G.: Static analysis for security. Secur. Priv. 2(6), 76–79 (2004). IEEE
Ayewah, N., Hovemeyer, D., Morgenthaler, J.D., et al.: Using static analysis to find bugs. Software 25(5), 22–29 (2008). IEEE
Nagappan, N., Ball, T.: Static analysis tools as early indicators of pre-release defect density. In: ACM 27th International Conference on Software Engineering, pp. 580–586. ACM (2005)
Zitser, M., Lippmann, R., Leek, T.: Testing static analysis tools using exploitable buffer overflows from open source code. ACM SIGSOFT Softw. Eng. Not. 29(6), 97–106 (2004). ACM
Ball, T.: The concept of dynamic analysis. In: Wang, J., Lemoine, M. (eds.) ESEC 1999 and ESEC-FSE 1999. LNCS, vol. 1687, pp. 216–234. Springer, Heidelberg (1999)
Mock, M.: Dynamic analysis from the bottom up. In: WODA 2003 ICSE Workshop on Dynamic Analysis, p. 13 (2003)
Ernst, M.D.: Static and dynamic analysis: synergy and duality. In: WODA 2003: ICSE Workshop on Dynamic Analysis, pp. 24–27 (2003)
Clarke, T.: Fuzzing for software vulnerability discovery. Department of Mathematic, Royal Holloway, University of London. Technical report. RHUL-MA-2009-4 (2009)
Yang, Q., Li, J.J., Weiss, D.M.: A survey of coverage-based testing tools. Comput. J. 52(5), 589–597 (2005)
Crawler4j - Open Source Web Crawler for Java. https://github.com/yasserg/crawler4j
Oehlert, P.: Violating assumptions with fuzzing. IEEE Secur. Priv. 3(2), 58–62 (2005)
Clarke, T., Crampton, J.: Fuzzing or how to help computers cope with the unexpected. Technical report, Royal Holloway University of London (2009)
Aitel, D.: The advantages of block-based protocol analysis for security testing. Technical report, Immunity Inc. (2002)
Juranic, L.: Using fuzzing to detect security vulnerabilities. Technical report, Infigo Information Security (2006)
Goodman, E.D.: Introduction to genetic algorithms. In: GECCO Conference Companion on Genetic and Evolutionary Computation, pp. 3205–3224. GECCO (2007)
Goldberg, D.E., Deb, K.: A comparative analysis of selection schemes used in genetic algorithms. In: Foundations of Genetic Algorithms, pp. 69–93 (1991)
Gen, M., Cheng, R.: Genetic Algorithms and Engineering Optimization. Wiley, New York (2000)
Deep, K., Mebrahtu, H.: Combined mutation operators of genetic algorithm for the travelling salesman problem. Int. J. Comb. Opt. Prob. Inf. 2(3), 1–23 (2011)
Srinivas, M., Patnaik, L.M.: Genetic algorithms: a survey. IEEE Comput. 27, 17–26 (1994). IEEE
Srinivas, M., Patnaik, L.M.: Adaptive probabilities of crossover and mutation in genetic algorithms. IEEE Trans. Syst. Man Cybern. 24(4), 656–667 (1994). IEEE
Fortin, F., De Rainville, F., et al.: DEAP: evolutionary algorithms made easy. J. Mach. Learn. Res. 13(1), 2171–2175 (2012)
Sulley: A Pure Python Fully-Automated and Unattended Fuzzing Framework. https://github.com/OpenRCE/sulley
Emma, A Free Java Code Coverage Tool. http://emma.sourceforge.net/
FindBugs - Find Bugs in Java Programs. http://findbugs.sourceforge.net/
Marovic, B., Wrzos, M., Lewandowski, M., et al.: GN3 quality assurance best practice guide 4.0. Technical report (2012)
Guang-Hong, L., Gang, W., Tao, Z., et al.: Vulnerability analysis for x86 executables using genetic algorithm and fuzzing. In: Third International Conference on Convergence and Hybrid Information Technology, IEEE ICCIT 2008, vol. 2, pp. 491–497 (2008)
Iozzo, V.: 0-knowledge fuzzing. Technical report, Black Hat DC (2010)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2015 Springer International Publishing Switzerland
About this paper
Cite this paper
Shortt, C., Weber, J. (2015). Hermes: A Targeted Fuzz Testing Framework. In: Fujita, H., Guizzi, G. (eds) Intelligent Software Methodologies, Tools and Techniques. SoMeT 2015. Communications in Computer and Information Science, vol 532. Springer, Cham. https://doi.org/10.1007/978-3-319-22689-7_35
Download citation
DOI: https://doi.org/10.1007/978-3-319-22689-7_35
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-22688-0
Online ISBN: 978-3-319-22689-7
eBook Packages: Computer ScienceComputer Science (R0)