Olli Saarikivi
Skip Abstract Section
Abstract
This article focuses on computing minimal test suites for multithreaded programs. Based on previous work on test case generation for multithreaded programs using unfoldings, this article shows how this unfolding can be used to generate minimal test suites covering all local states of the program. Generating such minimal test suites is shown to be NP-complete in the size of the unfolding. We propose an SMT encoding for this problem and two methods based on heuristics which only approximate the solution, but scale better in practice. Finally, we apply our methods to compute the minimal test suites for several benchmarks.
- Parosh Aziz Abdulla, Stavros Aronis, Mohamed Faouzi Atig, Bengt Jonsson, Carl Leonardsson, and Konstantinos F. Sagonas. 2015. Stateless model checking for TSO and PSO. In Proceedings of the 21st International Conference on Tools and Algorithms for the Construction and Analysis of Systems (TACAS 2015). Lecture Notes in Computer Science, Vol. 9035. Springer, 353--367. Google Scholar
Digital Library
- Parosh Aziz Abdulla, Stavros Aronis, Bengt Jonsson, and Konstantinos F. Sagonas. 2014. Optimal dynamic partial order reduction. In Proceedings of the 41st Annual ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages (POPL’14). ACM, 373--384. Google ScholarDigital Library
- Parosh Aziz Abdulla, Mohamed Faouzi Atig, Bengt Jonsson, and Carl Leonardsson. 2016. Stateless model checking for POWER. In Proceedings of the 28th International Conference on Computer Aided Verification (CAV’16). Lecture Notes in Computer Science, Vol. 9780. Springer, 134--156. Google ScholarCross Ref
- Paolo Baldan, Andrea Corradini, and Ugo Montanari. 2001. Contextual petri nets, asymmetric event structures, and processes. Inf. Comput. 171, 1 (2001), 1--49. Google ScholarDigital Library
- Sébastien Bornot, Rémi Morin, Peter Niebert, and Sarah Zennou. 2002. Black box unfolding with local first search. In Proceedings of the 8th International Conference on Tools and Algorithms for the Construction and Analysis of Systems (TACAS’02). Lecture Notes in Computer Science, Vol. 2280. Springer, 386--400. Google ScholarDigital Library
- Leonardo Mendonça de Moura and Nikolaj Bjørner. 2008. Z3: An efficient SMT solver. In Proceedings of the 14th International Conference on Tools and Algorithms for the Construction and Analysis of Systems (TACAS ’08). Lecture Notes in Computer Science, Vol. 4963. Springer, 337--340. Google ScholarDigital Library
- Volker Diekert and Grzegorz Rozenberg (Eds.). 1995. The Book of Traces. World Scientific Publishing Co., Inc. Google ScholarDigital Library
- Javier Esparza and Keijo Heljanko. 2008. Unfoldings—Partial-Order Approach to Model Checking. Springer. Google ScholarDigital Library
- Azadeh Farzan, Andreas Holzer, Niloofar Razavi, and Helmut Veith. 2013. Con2colic testing. In Joint Meeting of the European Software Engineering Conference and the ACM SIGSOFT Symposium on the Foundations of Software Engineering (ESEC/FSE’13). ACM, 37--47. Google ScholarDigital Library
- Azadeh Farzan and P. Madhusudan. 2006. Causal atomicity. In Proceedings of the 18th International Conference on Computer Aided Verification (CAV’06). Lecture Notes in Computer Science, Vol. 4144. Springer, 315--328. Google ScholarDigital Library
- Cormac Flanagan and Patrice Godefroid. 2005. Dynamic partial-order reduction for model checking software. In Proceedings of the 32nd ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages (POPL’05). ACM, 110--121. Google ScholarDigital Library
- Patrice Godefroid. 1996. Partial-Order Methods for the Verification of Concurrent Systems—An Approach to the State-Explosion Problem. Lecture Notes in Computer Science, Vol. 1032. Springer. Google ScholarDigital Library
- Patrice Godefroid, Nils Klarlund, and Koushik Sen. 2005. DART: Directed automated random testing. In Proceedings of the ACM SIGPLAN 2005 Conference on Programming Language Design and Implementation. ACM, 213--223. Google ScholarDigital Library
- Kari Kähkönen. 2015. Automated Systematic Testing Methods for Multithreaded Programs. Doctoral Dissertation. School of Science, Aalto University.Google Scholar
- Kari Kähkönen and Keijo Heljanko. 2014a. Lightweight state capturing for automated testing of multithreaded programs. In Proceedings of the 8th International Conference on Tests and Proofs (TAP’14). Lecture Notes in Computer Science, Vol. 8570. Springer, 187--203. Google ScholarCross Ref
- Kari Kähkönen and Keijo Heljanko. 2014b. Testing multithreaded programs with contextual unfoldings and dynamic symbolic execution. In Proceedings of the 14th International Conference on Application of Concurrency to System Design (ACSD’14). IEEE Computer Society, 142--151. Google ScholarDigital Library
- Kari Kähkönen, Olli Saarikivi, and Keijo Heljanko. 2012. Using unfoldings in automated testing of multithreaded programs. In Proceedings of the IEEE/ACM International Conference on Automated Software Engineering (ASE’12). ACM, 150--159. Google ScholarDigital Library
- Victor Khomenko, Alex Kondratyev, Maciej Koutny, and Walter Vogler. 2006. Merged processes: A new condensed representation of petri net behaviour. Acta Inf. 43, 5 (2006), 307--330. Google ScholarDigital Library
- Kenneth L. McMillan. 1995. A technique of state space search based on unfolding. Formal Methods Syst. Des. 6, 1 (1995), 45--65. Google ScholarDigital Library
- Ugo Montanari and Francesca Rossi. 1995. Contextual nets. Acta Inf. 32, 6 (1995), 545--596. Google ScholarDigital Library
- Peter Niebert, Michaela Huhn, Sarah Zennou, and Denis Lugiez. 2001. Local first search - a new paradigm for partial order reductions. In Proceedings of the 12th International Conference on Concurrency Theory (CONCUR’01). Lecture Notes in Computer Science, Vol. 2154. Springer, 396--410. Google ScholarDigital Library
- Mogens Nielsen, Gordon D. Plotkin, and Glynn Winskel. 1981. Petri nets, event structures and domains, part i. Theor. Comput. Sci. 13 (1981), 85--108. Google ScholarCross Ref
- Sampo Niskanen and Patric Östergård. 2002. Cliquer - routines for clique searching. Retrieved from http://users.aalto.fi/∼pat/cliquer.html.Google Scholar
- Patric Östergård. 2002. A fast algorithm for the maximum clique problem. Discr. Appl. Math. 120, 1--3 (2002), 197--207. Google ScholarDigital Library
- Hernán Ponce de León and Andrey Mokhov. 2015. Building bridges between sets of partial orders. In Proceedings of the 9th International Conference on Language and Automata Theory and Applications (LATA’15). Lecture Notes in Computer Science, Vol. 8977. Springer, 145--160. Google ScholarCross Ref
- Hernán Ponce de León, Olli Saarikivi, Kari Kähkönen, Keijo Heljanko, and Javier Esparza. 2015. Unfolding based minimal test suites for testing multithreaded programs. In Proceedings of the 15th International Conference on Application of Concurrency to System Design (ACSD’15). IEEE Computer Society, 40--49. Google ScholarDigital Library
- César Rodríguez. 2013. Verification Based on Unfoldings of Petri Nets with Read Arcs. Thèse de doctorat. Laboratoire Spécification et Vérification, ENS Cachan, France.Google Scholar
- César Rodríguez, Marcelo Sousa, Subodh Sharma, and Daniel Kroening. 2015. Unfolding-based partial order reduction. In Proceedings of the 26th International Conference on Concurrency Theory (CONCUR’15). 456--469.Google Scholar
- Koushik Sen. 2006. Scalable Automated Methods for Dynamic Program Analysis. Ph.D. Thesis. University of Illinois. Google ScholarDigital Library
- Antti Valmari. 1996. The state explosion problem. In Lectures on Petri Nets I: Basic Models, Advances in Petri Nets, the Volumes Are Based on the Advanced Course on Petri Nets. 429--528. Google ScholarDigital Library
Index Terms
- Minimizing Test Suites with Unfoldings of Multithreaded Programs
Recommendations
Unfolding Based Minimal Test Suites for Testing Multithreaded Programs
ACSD '15: Proceedings of the 2015 15th International Conference on Application of Concurrency to System DesignThis paper focuses on the problem of computing the minimal test suite for a terminating multithreaded program that covers all its executable statements. We have in previous work shown how to use unfoldings to capture the true concurrency semantics of ...
Prioritizing random combinatorial test suites
SAC '17: Proceedings of the Symposium on Applied ComputingThe behaviour of a system under test can be influenced by several factors, such as system configurations, user inputs, and so on. It has also been observed that many failures are caused by only a small number of factors. Combinatorial testing aims at ...
Adaptive random prioritization for interaction test suites
SAC '14: Proceedings of the 29th Annual ACM Symposium on Applied ComputingCombinatorial interaction testing (CIT), a black-box testing method, has been well studied in recent years. It aims at constructing an effective interaction test suites, so as to identify the faults that are caused by interactions among parameters. ...
Comments