Vilas Jagannath
ABSTRACT
Multithreaded code is notoriously hard to develop and test. A multithreaded test exercises the code under test with two or more threads. Each test execution follows some schedule/interleaving of the multiple threads, and different schedules can give different results. Developers often want to enforce a particular schedule for test execution, and to do so, they use time delays (Thread.sleep in Java). Unfortunately, this approach can produce false positives or negatives, and can result in unnecessarily long testing time.
This paper presents IMUnit, a novel approach to specifying and executing schedules for multithreaded tests. We introduce a new language that allows explicit specification of schedules as orderings on events encountered during test execution. We present a tool that automatically instruments the code to control test execution to follow the specified schedule, and a tool that helps developers migrate their legacy, sleep-based tests into event-based tests in IMUnit. The migration tool uses novel techniques for inferring events and schedules from the executions of sleep-based tests. We describe our experience in migrating over 200 tests. The inference techniques have high precision and recall of over 75%, and IMUnit reduces testing time compared to sleep-based tests on average 3.39x.
- A. V. Aho, M. R. Garey, and J. D. Ullman. The transitive reduction of a directed graph. SIAM Journal on Computing, 1972.Google Scholar
- R. Alur, P. Cerný, M. Parthasarathy, and W. Nam. Synthesis of interface specifications for Java classes. In POPL, 2005. Google ScholarDigital Library
- G. Ammons, R. Bodík, and J. R. Larus. Mining specifications. In POPL, 2002. Google ScholarDigital Library
- Apache Software Foundation. Apache Commons Collections. http://commons.apache.org/collections/.Google Scholar
- Apache Software Foundation. Apache Commons Pool. http://commons.apache.org/pool/.Google Scholar
- Apache Software Foundation. Apache Lucene. http://lucene.apache.org/.Google Scholar
- Apache Software Foundation. Apache MINA. http://mina.apache.org/.Google Scholar
- J. Burnim and K. Sen. DETERMIN: Inferring likely deterministic specifications of multithreaded programs. In ICSE, 2010. Google ScholarDigital Library
- R. H. Carver and K. Tai. Replay and testing for concurrent programs. IEEE Software, 1991. Google ScholarDigital Library
- F. Chen and G. Roşu. Mop: An efficient and generic runtime verification framework. In OOPSLA, 2007. Google ScholarDigital Library
- Codehaus. Sysunit. http://docs.codehaus.org/display/SYSUNIT/Home.Google Scholar
- S. Cotton. graphlib. http://www-verimag.imag.fr/cotton/.Google Scholar
- A. Dantas, F. V. Brasileiro, and W. Cirne. Improving automated testing of multi-threaded software. In ICST, 2008. Google ScholarDigital Library
- J. Huang, P. Liu, and C. Zhang. LEAP: Lightweight deterministic multi-processor replay of concurrent Java programs. In FSE, 2010. Google ScholarDigital Library
- V. Jagannath, M. Gligoric, D. Jin, G. Rosu, and D. Marinov. IMUnit: Improved multithreaded unit testing (position statement). In IWMSE, 2010. Google ScholarDigital Library
- V. Jagannath, Q. Luo, and D. Marinov. Change-aware preemption prioritization. In ISSTA, 2011. Google ScholarDigital Library
- Java Community Process. JSR 166: Concurrency utilities. http://g.oswego.edu/dl/concurrency-interest/.Google Scholar
- JBoss Community. JBoss Cache. http://www.jboss.org/jbosscache.Google Scholar
- G. Kiczales, E. Hilsdale, J. Hugunin, M. Kersten, J. Palm, and W. G. Griswold. An overview of AspectJ. In ECOOP, 2001. Google ScholarDigital Library
- Lassi Project. Sleep testcase. http://tinyurl.com/4hk9zdr.Google Scholar
- C. Lee, F. Chen, and G. Roşu. Mining parametric specifications. In ICSE, 2011. Google ScholarDigital Library
- B. Long, D. Hoffman, and P. A. Strooper. A concurrency test tool for Java monitors. In ASE, 2001. Google ScholarDigital Library
- B. Long, D. Hoffman, and P. A. Strooper. Tool support for testing concurrent Java components. IEEE TSE, 2003. Google ScholarDigital Library
- P. O. Meredith, D. Jin, D. Griffith, F. Chen, and G. Roşu. An overview of the MOP runtime verification framework. Springer STTT, 2011. Google ScholarDigital Library
- W. Pugh and N. Ayewah. MultithreadedTC - A framework for testing concurrent Java applications. http://code.google.com/p/multithreadedtc/.Google Scholar
- W. Pugh and N. Ayewah. Unit testing concurrent software. In ASE, 2007. Google ScholarDigital Library
- M. Ricken and R. Cartwright. ConcJUnit: Unit testing for concurrent programs. In PPPJ, 2009. Google ScholarDigital Library
- K. Tai and R. H. Carver. Use of sequencing constraints for specifying, testing, and debugging concurrent programs. In ICPADS, 1994. Google ScholarDigital Library
Index Terms
- Improved multithreaded unit testing
Recommendations
Smart Unit Testing Framework
ISSREW '12: Proceedings of the 2012 IEEE 23rd International Symposium on Software Reliability Engineering WorkshopsUnit testing(UT) is an important step in ensuring the quality of software. Considerable effort is spent in unit testing. There are several frameworks to help with UT. Some common frameworks are Cunit, Junit, Nunit etc. All of these have very similar ...
MuTMuT: Efficient Exploration for Mutation Testing of Multithreaded Code
ICST '10: Proceedings of the 2010 Third International Conference on Software Testing, Verification and ValidationMutation testing is a method for measuring the quality of test suites. Given a system under test and a test suite, mutations are systematically inserted into the system, and the test suite is executed to determine which mutants it detects. A major cost ...
Parameterized unit testing: theory and practice
ICSE '10: Proceedings of the 32nd ACM/IEEE International Conference on Software Engineering - Volume 2Unit testing has been widely recognized as an important and valuable means of improving software reliability, as it exposes bugs early in the software development life cycle. However, manual unit testing is often tedious and insufficient. Testing tools ...
Comments