research-article

Synthesizing tests for detecting atomicity violations

Authors:

Malavika Samak,

Murali Krishna RamanathanAuthors Info & Claims

ESEC/FSE 2015: Proceedings of the 2015 10th Joint Meeting on Foundations of Software Engineering

Pages 131 - 142

https://doi.org/10.1145/2786805.2786874

Published: 30 August 2015 Publication History

Abstract

Using thread-safe libraries can help programmers avoid the complexities of multithreading. However, designing libraries that guarantee thread-safety can be challenging. Detecting and eliminating atomicity violations when methods in the libraries are invoked concurrently is vital in building reliable client applications that use the libraries. While there are dynamic analyses to detect atomicity violations, these techniques are critically dependent on effective multithreaded tests. Unfortunately, designing such tests is non-trivial. In this paper, we design a novel and scalable approach for synthesizing multithreaded tests that help detect atomicity violations. The input to the approach is the implementation of the library and a sequential seed testsuite that invokes every method in the library with random parameters. We analyze the execution of the sequential tests, generate variable lock dependencies and construct a set of three accesses which when interleaved suitably in a multithreaded execution can cause an atomicity violation. Subsequently, we identify pairs of method invocations that correspond to these accesses and invoke them concurrently from distinct threads with appropriate objects to help expose atomicity violations. We have incorporated these ideas in our tool, named Intruder, and applied it on multiple open-source Java multithreaded libraries. Intruder is able to synthesize 40 multithreaded tests across nine classes in less than two minutes to detect 79 harmful atomicity violations, including previously unknown violations in thread-safe classes. We also demonstrate the effectiveness of Intruder by comparing the results with other approaches designed for synthesizing multithreaded tests.

References

[1]

N. Ayewah and W. Pugh. The Google FindBugs Fixit. In Proceedings of the 19th International Symposium on Software Testing and Analysis, ISSTA ’10, pages 241–252, 2010.

Digital Library

[2]

S. Biswas, J. Huang, A. Sengupta, and M. D. Bond. Doublechecker: E fficient sound and precise atomicity checking. In Proceedings of the 35th ACM SIGPLAN Conference on Programming Language Design and Implementation, PLDI ’14, pages 28–39, 2014.

Digital Library

[3]

J. Bogda and U. Hölzle. Removing unnecessary synchronization in Java. In Proceedings of the 14th ACM SIGPLAN Conference on Object-oriented Programming, Systems, Languages, and Applications, OOPSLA ’99, pages 35–46, 1999.

Digital Library

[4]

C. Cadar, D. Dunbar, and D. Engler. KLEE: Unassisted and automatic generation of high-coverage tests for complex systems programs. In Proceedings of the 8th USENIX Conference on Operating Systems Design and Implementation, OSDI’08, pages 209–224, 2008.

Digital Library

[5]

C. Cadar and D. Engler. Execution generated test cases: How to make systems code crash itself. In Proceedings of the 12th International Conference on Model Checking Software, SPIN’05, pages 2–23, 2005.

Digital Library

[6]

C. Flanagan and S. N. Freund. Fasttrack: E fficient and precise dynamic race detection. In Proceedings of the 2009 ACM SIGPLAN Conference on Programming Language Design and Implementation, PLDI ’09, 2009.

Digital Library

[7]

C. Flanagan, S. N. Freund, and J. Yi. Velodrome: A sound and complete dynamic atomicity checker for multithreaded programs. In Proceedings of the 2008 ACM SIGPLAN Conference on Programming Language Design and Implementation, PLDI ’08.

Digital Library

[8]

C. Flanagan and S. Qadeer. A type and e ffect system for atomicity. In Proceedings of the ACM SIGPLAN 2003 Conference on Programming Language Design and Implementation, PLDI ’03, pages 338–349, 2003.

Digital Library

[9]

G. Fraser and A. Arcuri. EvoSuite: Automatic test suite generation for object-oriented software. In Proceedings of the 19th ACM SIGSOFT Symposium and the 13th European Conference on Foundations of Software Engineering, ESEC /FSE ’11, pages 416–419, 2011.

Digital Library

[10]

P. Godefroid, N. Klarlund, and K. Sen. Dart: Directed automated random testing. In Proceedings of the 2005 ACM SIGPLAN Conference on Programming Language Design and Implementation, PLDI ’05, 2005.

Digital Library

[11]

M. P. Herlihy and J. M. Wing. Linearizability: A correctness condition for concurrent objects. ACM Trans. Program. Lang. Syst., 12(3):463–492, July 1990.

Digital Library

[12]

V. Jagannath, M. Gligoric, D. Jin, Q. Luo, G. Rosu, and D. Marinov. Improved multithreaded unit testing. In Proceedings of the 19th ACM SIGSOFT Symposium and the 13th European Conference on Foundations of Software Engineering, ESEC /FSE ’11, pages 223–233, 2011.

Digital Library

[13]

P. Joshi, C.-S. Park, K. Sen, and M. Naik. A randomized dynamic program analysis technique for detecting real deadlocks. In Proceedings of the 2009 ACM SIGPLAN Conference on Programming Language Design and Implementation, PLDI ’09, 2009.

Digital Library

[14]

R. J. Lipton. Reduction: A method of proving properties of parallel programs. Commun. ACM, 18(12):717–721, Dec. 1975.

Digital Library

[15]

P. Liu, O. Tripp, and X. Zhang. Flint: Fixing linearizability violations. In Proceedings of the 2014 ACM International Conference on Object Oriented Programming Systems Languages and Applications, OOPSLA ’14, pages 543–560, 2014.

Digital Library

[16]

S. Lu, S. Park, C. Hu, X. Ma, W. Jiang, Z. Li, R. A. Popa, and Y. Zhou. MUVI: Automatically inferring multi-variable access correlations and detecting related semantic and concurrency bugs. In Proceedings of Twenty-first ACM SIGOPS Symposium on Operating Systems Principles, SOSP ’07, pages 103–116, 2007.

Digital Library

[17]

S. McPeak, C. H. Gros, and M. K. Ramanathan. Scalable and incremental software bug detection. In Proceedings of the 2013 9th Joint Meeting on Foundations of Software Engineering, ESEC /FSE 2013.

Digital Library

[18]

M. Musuvathi and S. Qadeer. Iterative context bounding for systematic testing of multithreaded programs. In Proceedings of the 2007 ACM SIGPLAN Conference on Programming Language Design and Implementation, PLDI ’07, 2007.

Digital Library

[19]

A. Nistor, Q. Luo, M. Pradel, T. R. Gross, and D. Marinov. Ballerina: Automatic generation and clustering of e fficient random unit tests for multithreaded code. In Proceedings of the 34th International Conference on Software Engineering (ICSE), pages 727–737. IEEE, 2012.

Digital Library

[20]

C. Pacheco, S. K. Lahiri, M. D. Ernst, and T. Ball. Feedback-directed random test generation. In Proceedings of the 29th International Conference on Software Engineering, ICSE ’07, pages 75–84, 2007.

Digital Library

[21]

C.-S. Park and K. Sen. Randomized active atomicity violation detection in concurrent programs. In Proceedings of the 16th ACM SIGSOFT International Symposium on Foundations of Software Engineering, SIGSOFT ’08 /FSE-16, pages 135–145, 2008.

Digital Library

[22]

S. Park, S. Lu, and Y. Zhou. CTrigger: Exposing atomicity violation bugs from their hiding places. In Proceedings of the 14th International Conference on Architectural Support for Programming Languages and Operating Systems, ASPLOS XIV, 2009.

Digital Library

[23]

M. Pradel and T. R. Gross. Fully automatic and precise detection of thread safety violations. In Proceedings of the 33rd ACM SIGPLAN Conference on Programming Language Design and Implementation, PLDI ’12, 2012.

Digital Library

[24]

W. Pugh and N. Ayewah. Unit testing concurrent software. In In ASE, 2007.

Digital Library

[25]

M. Samak and M. K. Ramanathan. Multithreaded test synthesis for deadlock detection. In Proceedings of the 2014 ACM International Conference on Object Oriented Programming Systems Languages and Applications, OOPSLA ’14, pages 473–489, 2014.

Digital Library

[26]

M. Samak and M. K. Ramanathan. Omen +: A precise dynamic deadlock detector for multithreaded java libraries. In Proceedings of the 22Nd ACM SIGSOFT International Symposium on Foundations of Software Engineering, FSE 2014, pages 735–738, 2014.

Digital Library

[27]

M. Samak, M. K. Ramanathan, and S. Jagannathan. Synthesizing racy tests. In Proceedings of the 36th ACM SIGPLAN Conference on Programming Language Design and Implementation, PLDI 2015, pages 175–185, 2015.

Digital Library

[28]

S. Savage, M. Burrows, G. Nelson, P. Sobalvarro, and T. Anderson. Eraser: a dynamic data race detector for multithreaded programs. ACM Trans. Comput. Syst., 15(4):391–411, Nov. 1997.

Digital Library

[29]

K. Sen. Race directed random testing of concurrent programs. In Proceedings of the 2008 ACM SIGPLAN Conference on Programming Language Design and Implementation, PLDI ’08, pages 11–21, 2008.

Digital Library

[30]

K. Sen, D. Marinov, and G. Agha. Cute: A concolic unit testing engine for c. In Proceedings of the 10th European Software Engineering Conference Held Jointly with 13th ACM SIGSOFT International Symposium on Foundations of Software Engineering, ESEC /FSE-13, pages 263–272, 2005.

Digital Library

[31]

F. Sorrentino, A. Farzan, and P. Madhusudan. Penelope: Weaving threads to expose atomicity violations. In Proceedings of the Eighteenth ACM SIGSOFT International Symposium on Foundations of Software Engineering, FSE ’10, pages 37–46, 2010.

Digital Library

[32]

S. Steenbuck and G. Fraser. Generating unit tests for concurrent classes. In Proceedings of the Sixth International Conference on Software Testing, Verification and Validation (ICST), pages 144–153. IEEE, 2013.

Digital Library

[33]

R. Vallee-Rai, E. Gagnon, L. Hendren, P. Lam, P. Pominville, and V. Sundaresan. Optimizing java bytecode using the soot framework: Is it feasible? In In International Conference on Compiler Construction, LNCS 1781, pages 18–34, 2000.

Digital Library

Cited By

Zhao YWang ZLiu SSun JChen JChen X(2023)Achieving High MAP-Coverage Through Pattern Constraint ReductionIEEE Transactions on Software Engineering10.1109/TSE.2022.314448049:1(99-112)Online publication date: 1-Jan-2023
https://doi.org/10.1109/TSE.2022.3144480
Chabbi MRamanathan MJhala RDillig I(2022)A study of real-world data races in GolangProceedings of the 43rd ACM SIGPLAN International Conference on Programming Language Design and Implementation10.1145/3519939.3523720(474-489)Online publication date: 9-Jun-2022
https://dl.acm.org/doi/10.1145/3519939.3523720
Gong SAltinbüken DFonseca PManiatis P(2021)SnowboardProceedings of the ACM SIGOPS 28th Symposium on Operating Systems Principles10.1145/3477132.3483549(66-83)Online publication date: 26-Oct-2021
https://dl.acm.org/doi/10.1145/3477132.3483549
Show More Cited By

Index Terms

Synthesizing tests for detecting atomicity violations
1. Software and its engineering
  1. Software creation and management
    1. Software verification and validation
      1. Software defect analysis
        Software testing and debugging
2. Theory of computation
  1. Semantics and reasoning
    1. Program semantics

Recommendations

Randomized active atomicity violation detection in concurrent programs
SIGSOFT '08/FSE-16: Proceedings of the 16th ACM SIGSOFT International Symposium on Foundations of software engineering

Atomicity is an important specification that enables programmers to understand atomic blocks of code in a multi-threaded program as if they are sequential. This significantly simplifies the programmer's job to reason about correctness. Several modern ...
Synthesizing racy tests
PLDI '15: Proceedings of the 36th ACM SIGPLAN Conference on Programming Language Design and Implementation

Subtle concurrency errors in multithreaded libraries that arise because of incorrect or inadequate synchronization are often difficult to pinpoint precisely using only static techniques. On the other hand, the effectiveness of dynamic race detectors is ...
Synthesizing racy tests
PLDI '15

Subtle concurrency errors in multithreaded libraries that arise because of incorrect or inadequate synchronization are often difficult to pinpoint precisely using only static techniques. On the other hand, the effectiveness of dynamic race detectors is ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

ESEC/FSE 2015: Proceedings of the 2015 10th Joint Meeting on Foundations of Software Engineering

August 2015

1068 pages

ISBN:9781450336758

DOI:10.1145/2786805

General Chair:
Elisabetta Di Nitto
Politecnico di Milano, Italy
,
Program Chairs:
Mark Harman
University College London, UK
,
Patrick Heymans
University of Namur, Belgium

Copyright © 2015 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGSOFT: ACM Special Interest Group on Software Engineering

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 30 August 2015

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

ESEC/FSE'15

Sponsor:

SIGSOFT

ESEC/FSE'15: Joint Meeting of the European Software Engineering Conference and the ACM SIGSOFT Symposium on the Foundations of Software Engineering

August 30 - September 4, 2015

Bergamo, Italy

Acceptance Rates

Overall Acceptance Rate 112 of 543 submissions, 21%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

31
Total Citations
View Citations
377
Total Downloads

Downloads (Last 12 months)14
Downloads (Last 6 weeks)1

Reflects downloads up to 25 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Zhao YWang ZLiu SSun JChen JChen X(2023)Achieving High MAP-Coverage Through Pattern Constraint ReductionIEEE Transactions on Software Engineering10.1109/TSE.2022.314448049:1(99-112)Online publication date: 1-Jan-2023
https://doi.org/10.1109/TSE.2022.3144480
Chabbi MRamanathan MJhala RDillig I(2022)A study of real-world data races in GolangProceedings of the 43rd ACM SIGPLAN International Conference on Programming Language Design and Implementation10.1145/3519939.3523720(474-489)Online publication date: 9-Jun-2022
https://dl.acm.org/doi/10.1145/3519939.3523720
Gong SAltinbüken DFonseca PManiatis P(2021)SnowboardProceedings of the ACM SIGOPS 28th Symposium on Operating Systems Principles10.1145/3477132.3483549(66-83)Online publication date: 26-Oct-2021
https://dl.acm.org/doi/10.1145/3477132.3483549
Khanna DPurandare RSharma S(2021)Synthesizing Multi-threaded Tests from Sequential Traces to Detect Communication Deadlocks2021 14th IEEE Conference on Software Testing, Verification and Validation (ICST)10.1109/ICST49551.2021.00013(1-12)Online publication date: Apr-2021
https://doi.org/10.1109/ICST49551.2021.00013
Du XYin LFeng HDong W(2021)Program Verification Enhanced Precise Analysis of Interrupt-Driven Program Vulnerabilities2021 28th Asia-Pacific Software Engineering Conference (APSEC)10.1109/APSEC53868.2021.00033(253-263)Online publication date: Dec-2021
https://doi.org/10.1109/APSEC53868.2021.00033
Terragni VPezzè M(2021)Statically driven generation of concurrent tests for thread‐safe classesSoftware Testing, Verification and Reliability10.1002/stvr.177431:4Online publication date: 4-May-2021
https://doi.org/10.1002/stvr.1774
Mathur UViswanathan MLarus JCeze LStrauss K(2020)Atomicity Checking in Linear Time using Vector ClocksProceedings of the Twenty-Fifth International Conference on Architectural Support for Programming Languages and Operating Systems10.1145/3373376.3378475(183-199)Online publication date: 9-Mar-2020
https://dl.acm.org/doi/10.1145/3373376.3378475
Khanna DPurandare RSharma S(2020)Verifying and Testing Concurrent Programs using Constraint Solver based Approaches2020 IEEE International Conference on Software Maintenance and Evolution (ICSME)10.1109/ICSME46990.2020.00105(834-838)Online publication date: Sep-2020
https://doi.org/10.1109/ICSME46990.2020.00105
Emmi MEnea C(2019)Weak-consistency specification via visibility relaxationProceedings of the ACM on Programming Languages10.1145/32903733:POPL(1-28)Online publication date: 2-Jan-2019
https://dl.acm.org/doi/10.1145/3290373
Terragni VPezze MBianchi F(2019)Coverage-Driven Test Generation for Thread-Safe Classes via Parallel and Conflict Dependencies2019 12th IEEE Conference on Software Testing, Validation and Verification (ICST)10.1109/ICST.2019.00034(264-275)Online publication date: Apr-2019
https://doi.org/10.1109/ICST.2019.00034
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten