ABSTRACT
Writing property-based testing models is a challenging task. This article introduces a new tool, Faktory, which is capable of automatically generating an executable property-based testing model from less complicated sources: normal function type specifications, and traditional function calling contracts using pre- and post-conditions. Concretely, Faktory is an Elixir library which from an API annotated with executable calling contracts written using the Corsa contract checking library, automatically generates a property-based testing model in the form of a state machine implemented using the Makina state machine DSL. In the article we illustrate the functionalities of the Faktory tool using a number of examples. The first one shows how to test a functional data structure; the second shows how to improve test-case generation by reusing test results; and in the third example a stateful key-value storage is tested by first deriving an initial test model using Faktory, and then refining the generated model using the Makina model/state machine extension mechanism.
- Klaus Alfert. 2015. https://github.com/alfert/propcheck Accessed: 2021-05-06 Google Scholar
- Thomas Arts, Laura M. Castro, and John Hughes. 2008. Testing Erlang Data Types with Quviq Quickcheck. In Proceedings of the 7th ACM SIGPLAN Workshop on ERLANG (ERLANG ’08). Association for Computing Machinery, New York, NY, USA. 1–8. isbn:9781605580654 https://doi.org/10.1145/1411273.1411275 Google ScholarDigital Library
- Luis Eduardo Bueso de Barrio, Lars-Åke Fredlund, Ángel Herranz, Clara Benac Earle, and Julio Mariño. 2021. Makina: A New QuickCheck State Machine Library. In Proceedings of the 20th ACM SIGPLAN International Workshop on Erlang (Erlang 2021). Association for Computing Machinery, New York, NY, USA. 41–53. isbn:9781450386128 https://doi.org/10.1145/3471871.3472964 Google ScholarDigital Library
- Laura M. Castro, Pablo Lamela, and S. Thompson. 2017. Making Property-Based Testing Easier to Read for Humans. COMPUTING AND INFORMATICS, 35, 4 (2017), Feb., 890–913. https://www.cai.sk/ojs/index.php/cai/article/view/3381 Google Scholar
- Koen Claessen and John Hughes. 2000. QuickCheck: A Lightweight Tool for Random Testing of Haskell Programs. In Proceedings of the Fifth ACM SIGPLAN International Conference on Functional Programming (ICFP ’00). ACM, New York, NY, USA. 268–279. isbn:1-58113-202-6 https://doi.org/10.1145/351240.351266 Google ScholarDigital Library
- Lars-Åke Fredlund, Clara Benac Earle, Ángel Herranz-Nieva, and Julio Mariño-Carballo. 2014. Property-Based Testing of JSON Based Web Services. In 2014 IEEE International Conference on Web Services, ICWS, 2014, Anchorage, AK, USA, June 27 - July 2, 2014. IEEE Computer Society, 704–707. https://doi.org/10.1109/ICWS.2014.110 Google ScholarCross Ref
- Alex Gerdes, John Hughes, Nicholas Smallbone, Stefan Hanenberg, Sebastian Ivarsson, and Meng Wang. 2018. Understanding formal specifications through good examples. In Proceedings of the 17th ACM SIGPLAN International Workshop on Erlang, ICFP 2018, St. Louis, MO, USA, September 23-29, 2018, Natalia Chechina and Adrian Francalanza (Eds.). ACM, 13–24. https://doi.org/10.1145/3239332.3242763 Google ScholarDigital Library
- Patrice Godefroid, Adam Kiezun, and Michael Y. Levin. 2008. Grammar-based whitebox fuzzing. In Proceedings of the ACM SIGPLAN 2008 Conference on Programming Language Design and Implementation, Tucson, AZ, USA, June 7-13, 2008, Rajiv Gupta and Saman P. Amarasinghe (Eds.). ACM, 206–215. isbn:978-1-59593-860-2 https://doi.org/10.1145/1375581.1375607 Google ScholarDigital Library
- Patrice Godefroid, Nils Klarlund, and Koushik Sen. 2005. DART: directed automated random testing. In Proceedings of the ACM SIGPLAN 2005 Conference on Programming Language Design and Implementation, Chicago, IL, USA, June 12-15, 2005, Vivek Sarkar and Mary W. Hall (Eds.). ACM, 213–223. isbn:1-59593-056-6 https://doi.org/10.1145/1065010.1065036 Google ScholarDigital Library
- A. Hellesøy. 2008. https://cucumber.io/ Accessed: 2023-05-11 Google Scholar
- John Hughes, Benjamin C. Pierce, Thomas Arts, and Ulf Norell. 2016. Mysteries of DropBox: Property-Based Testing of a Distributed Synchronization Service. In 2016 IEEE International Conference on Software Testing, Verification and Validation (ICST). 135–145. https://doi.org/10.1109/ICST.2016.37 Google ScholarCross Ref
- John M. Hughes and Hans Bolinder. 2011. Testing a Database for Race Conditions with QuickCheck. In Proceedings of the 10th ACM SIGPLAN Workshop on Erlang (Erlang ’11). Association for Computing Machinery, New York, NY, USA. 72–77. isbn:9781450308595 https://doi.org/10.1145/2034654.2034667 Google ScholarDigital Library
- Bertrand Meyer. 1992. Applying "Design by Contract". IEEE Computer, 25, 10 (1992), 40–51. https://doi.org/10.1109/2.161279 Google ScholarDigital Library
- Manolis Papadakis and Konstantinos Sagonas. 2011. A PropEr Integration of Types and Function Specifications with Property-Based Testing. In Proceedings of the 10th ACM SIGPLAN Workshop on Erlang (Erlang ’11). Association for Computing Machinery, New York, NY, USA. 39–50. isbn:9781450308595 https://doi.org/10.1145/2034654.2034663 Google ScholarDigital Library
- Sergio Pérez, Luis Eduardo Bueso de Barrio, Ignacio Ballesteros, Ángel Herranz, Julio Mariño, Clara Benac Earle, and Lars-Åke Fredlund. 2022. Executable Contracts for Elixir. In Proceedings of the 21st ACM SIGPLAN International Workshop on Erlang (Erlang 2022). Association for Computing Machinery, New York, NY, USA. 40–46. isbn:9781450394352 https://doi.org/10.1145/3546186.3549930 Google ScholarDigital Library
- Nick Stephens, John Grosen, Christopher Salls, Andrew Dutcher, Ruoyu Wang, Jacopo Corbetta, Yan Shoshitaishvili, Christopher Krügel, and Giovanni Vigna. 2016. Driller: Augmenting Fuzzing Through Selective Symbolic Execution. In Network and Distributed System Security Symposium. https://doi.org/10.14722/ndss.2016.23368 Google ScholarCross Ref
- Wiebe-Marten Wijnja. 2022. TypeCheck - Effortless runtime type-checking. https://github.com/Qqwy/elixir-type_check In Elixir EU 2022. Google Scholar
- Michal Zalewski. 2017. American fuzzy lop. http://lcamtuf.coredump.cx/afl/ Accessed: 2019-05-22 Google Scholar
Index Terms
- Generation and Refinement of Testing Models
Recommendations
Automatic generation of test models and properties from UML models with OCL constraints
OCL '12: Proceedings of the 12th Workshop on OCL and Textual ModellingModel-Based Testing and Property-Based Testing are two testing methodologies that usually facilitate the automation of the generation of test cases, using either models or properties as basis to derive complete test suites. In doing so, they also ...
On automatic generation of RTL validation test benches using circuit testing techniques
GLSVLSI '03: Proceedings of the 13th ACM Great Lakes symposium on VLSIIn this paper, we examine how good validation test benches can be automatically generated starting from the RTL description of a circuit. We develop our methodology based on extensive experiments performed with several popular benchmarks as well as ...
Applying “Functional Scenario-Based” Test Case Generation Method in Unit Testing and Integration Testing
Structured Object-Oriented Formal Language and MethodAbstractSpecification-based testing enables us to detect errors in the implementation of functions defined in given specifications. Its effectiveness in achieving high path coverage and efficiency in generating test cases are always major concerns of ...
Comments