Aggelos Giantsios
Nikolaos Papaspyrou
ABSTRACT
Concolic testing is a software testing technique combining concrete execution of a program (given specific input, along specific paths) with symbolic execution (generating new test inputs that give better path coverage than random test case generation). Concolic testing has so far been applied, mainly at the level of bytecode or assembly code, to programs written in imperative languages that manipulate primitive data types such as integers and arrays. In this paper, we demonstrate its application to a functional programming language core, a subset of the core language of Erlang, that supports pattern matching, structured recursive data types such as lists, recursion and higher-order functions. Moreover, we present CutEr, a tool implementing this testing technique. We describe CutEr's architecture, the challenges that need to be addressed by such a tool, its current limitations, and report some experiences from its use.
- J. Armstrong. Erlang. Commun. ACM, 53(9):68--75, 2010. DOI: 10.1145/1810891.1810910. Google Scholar
Digital Library
- C. Barrett, A. Stump, and C. Tinelli. The SMT-LIB Standard: Version 2.0. In A. Gupta and D. Kroening, editors, Proceedings of the 8th International Workshop on Satisfiability Modulo Theories, 2010.Google Scholar
- C. Cadar, D. Dunbar, and D. Engler. KLEE: Unassisted and automatic generation of high-coverage tests for complex systems programs. In Proceedings of the 8th USENIX Conference on Operating Systems Design and Implementation, pages 209--224, Berkeley, CA, USA, 2008. USENIX Association. URL: http://dl.acm.org/citation.cfm?id=1855741.1855756. Google ScholarDigital Library
- R. Carlsson and M. Rémond. EUnit: A lightweight unit testing framework for Erlang. In Proceedings of the 2006 ACM SIGPLAN Workshop on Erlang, pages 1--1, New York, NY, USA, 2006. ACM. ISBN 1-59593-490-1. DOI: 10.1145/1159789.1159791. Google ScholarDigital Library
- R. Carlsson, T. Lindgren, B. Gustavsson, S.-O. Nyström, R. Virding, E. Johansson, and M. Pettersson. Core Erlang 1.0.3 language specification. Technical report, Uppsala University, Nov. 2004. URL: https://www.it.uu.se/research/group/hipe/cerl/doc/core_erlang-1.0.3.pdf.Google Scholar
- K. Claessen and J. Hughes. QuickCheck: A lightweight tool for random testing of haskell programs. In Proceedings of the Fifth ACM SIGPLAN International Conference on Functional Programming, pages 268--279, New York, NY, USA, 2000. ACM. DOI: 10.1145/351240.351266. Google ScholarDigital Library
- L. De Moura and N. Bjørner. Z3: An efficient SMT solver. In Proceedings of the Theory and Practice of Software, 14th International Conference on Tools and Algorithms for the Construction and Analysis of Systems, pages 337--340, Berlin, Heidelberg, 2008. Springer-Verlag. DOI: 10.1007/978-3-540-78800-3_24. Google ScholarDigital Library
- P. Godefroid, N. Klarlund, and K. Sen. DART: directed automated random testing. In Proceedings of the ACM SIGPLAN Conference on Programming Language Design and Implementation, pages 213--223. ACM Press, June 2005. DOI: 10.1145/1065010.1065036. Google ScholarDigital Library
- P. Godefroid, M. Y. Levin, and D. Molnar. SAGE: Whitebox fuzzing for security testing. Commun. ACM, 55(3):40--44, Mar. 2012. DOI: 10.1145/2093548.2093564. Google ScholarDigital Library
- E. Larson and T. Austin. High coverage detection of input-related security faults. In Proceedings of the 12th USENIX Security Symposium. USENIX Association, Aug. 2003. URL: http://dl.acm.org/citation.cfm?id=1251353.1251362. Google ScholarDigital Library
- T. Lindahl and K. Sagonas. TypEr: A type annotator of Erlang code. In Proceedings of the 2005 ACM SIGPLAN Workshop on Erlang, pages 17--25, New York, NY, USA, 2005. ACM. DOI: 10.1145/1088361.1088366. Google ScholarDigital Library
- T. Lindahl and K. Sagonas. Practical type inference based on success typings. In Proceedings of the 8th ACM SIGPLAN Symposium on Principles and Practice of Declarative Programming, pages 167--178, New York, NY, USA, 2006. ACM Press. DOI: 10.1145/1140335.1140356. Google ScholarDigital Library
- M. Papadakis and K. Sagonas. A PropEr integration of types and function specifications with property-based testing. In Proceedings of the 10th ACM SIGPLAN Workshop on Erlang, pages 39--50, New York, NY, USA, 2011. ACM. DOI: 10.1145/2034654.2034663. Google ScholarDigital Library
- N. Papaspyrou and K. Sagonas. On preserving term sharing in the Erlang virtual machine. In Proceedings of the Eleventh ACM SIGPLAN Workshop on Erlang Workshop, pages 11--20, New York, NY, USA, 2012. ACM. DOI: 10.1145/2364489.2364493. Google ScholarDigital Library
- C. S. Păsăreanu, N. Rungta, and W. Visser. Symbolic execution with mixed concrete-symbolic solving. In Proceedings of the 2011 International Symposium on Software Testing and Analysis, pages 34--44, New York, NY, USA, 2011. ACM. DOI: 10.1145/2001420.2001425. Google ScholarDigital Library
- QuviQ. Erlang QuickCheck. URL: http://www.quviq.com/products/erlang-quickcheck/.Google Scholar
- C. Runciman, M. Naylor, and F. Lindblad. SmallCheck and Lazy Smallcheck: Automatic exhaustive testing for small values. In Proceedings of the First ACM SIGPLAN Symposium on Haskell, pages 37--48, New York, NY, USA, 2008. ACM. DOI: 10.1145/1411286.1411292. Google ScholarDigital Library
- K. Sen, D. Marinov, and G. Agha. CUTE: A concolic unit testing engine for C. In Proceedings of the Fifth Joint Meeting of the European Software Engineering Conference and ACM SIGSOFT Symposium on the Foundations of Software Engineering, pages 263--272. ACM Press, 2005. DOI: 10.1145/1081706.1081750. Google ScholarDigital Library
- G. Vidal. Concolic execution and test case generation in Prolog. In Proceedings of the 24th International Symposium on Logic-Based Program Synthesis and Transformation, volume 8981 of LNCS, pages 167--181. Springer, 2015. DOI: 10.1007/978-3-319-17822-6_10.Google ScholarCross Ref
Index Terms
- Concolic testing for functional languages
Recommendations
Concolic testing
ASE '07: Proceedings of the 22nd IEEE/ACM International Conference on Automated Software EngineeringConcolic testing automates test input generation by combining the concrete and symbolic (concolic) execution of the code under test. Traditional test input generation techniques use either (1) concrete execution or (2) symbolic execution that builds ...
Testing domain-specific languages
OOPSLA '11: Proceedings of the ACM international conference companion on Object oriented programming systems languages and applications companionThe Spoofax testing language provides a new approach to testing domain-specific languages as they are developed. It allows test cases to be written using fragments of the language under test, providing full IDE support for writing test cases and ...
Grey-box concolic testing on binary code
ICSE '19: Proceedings of the 41st International Conference on Software EngineeringWe present grey-box concolic testing, a novel path-based test case generation method that combines the best of both white-box and grey-box fuzzing. At a high level, our technique systematically explores execution paths of a program under test as in ...
Comments