Lin Tan
Yuanyuan Zhou
ABSTRACT
Concurrency bugs in an operating system (OS) are detrimental as they can cause the OS to fail and affect all applications running on top of the OS. Detecting OS concurrency bugs is challenging due to the complexity of the OS synchronization, particularly with the presence of the OS specific interrupt context. Existing dynamic concurrency bug detection techniques are designed for user level applications and cannot be applied to operating systems.
To detect OS concurrency bugs, we proposed a new type of annotations - interrupt related annotations - and generated 96,821 such annotations for the Linux kernel with little manual effort. These annotations have been used to automatically detect 9 real OS concurrency bugs (7 of which were previously unknown). Two of the key techniques that make the above contributions possible are: (1) using a hybrid approach to extract annotations from both code and comments written in natural language to achieve better coverage and accuracy in annotation extraction and bug detection; and (2) automatically propagating annotations to caller functions to improve annotating and bug detection. These two techniques are general and can be applied to non-OS code, code written in other programming languages such as Java, and for extracting other types of specifications.
- Ghostdoc. http://submain.com/products/ghostdoc.aspx.Google Scholar
- Runtime locking correctness validator. http://www.mjmwired.net/kernel/Documentation/lockdep-design.txt.Google Scholar
- Z. Anderson, D. Gay, R. Ennals, and E. Brewer. SharC: Checking data sharing strategies for multithreaded C. In PLDI, 2008. Google ScholarDigital Library
- T. Ball, B. Hackett, S. Lahiri, and S. Qadeer. Annotation-based property checking for systems software. Research report MSR-TR-2008-82, Microsoft Research, May 2008.Google Scholar
- M. D. Bond, K. E. Coons, and K. S. McKinley. PACER: Proportional detection of data races. In PLDI, 2010. Google ScholarDigital Library
- L. C. Briand, Y. Labiche, and X. Liu. Using machine learning to support debugging with Tarantula. In ISSRE, 2007. Google ScholarDigital Library
- J. Burnim and K. Sen. DETERMIN: Inferring likely deterministic specifications of multithreaded programs. In ICSE, 2010. Google ScholarDigital Library
- J.-D. Choi, K. Lee, A. Loginov, R. O'Callahan, V. Sarkar, and M. Sridharan. Efficient and precise datarace detection for multithreaded object-oriented programs. In PLDI, 2002. Google ScholarDigital Library
- J. Condit, M. Harren, Z. R. Anderson, D. Gay, and G. C. Necula. Dependent types for low-level programming. In ESOP, 2007. Google ScholarDigital Library
- J. Corbet, A. Rubini, and G. Kroah-Hartman. Linux Device Drivers, Third Edition. Reilly, 2005. Google ScholarDigital Library
- E. W. Dijkstra. The structure of the "THE"-multiprogramming system. In SOSP, 1967. Google ScholarDigital Library
- D. R. Engler and K. Ashcraft. RacerX: Effective, static detection of race conditions and deadlocks. In SOSP, 2003. Google ScholarDigital Library
- D. R. Engler, D. Y. Chen, S. Hallem, A. Chou, and B. Chelf. Bugs as deviant behavior: A general approach to inferring errors in systems code. In SOSP, 2001. Google ScholarDigital Library
- M. D. Ernst, A. Czeisler, W. G. Griswold, and D. Notkin. Quickly detecting relevant program invariants. In ICSE, 2000. Google ScholarDigital Library
- D. Evans. Static detection of dynamic memory errors. In PLDI, 1996. Google ScholarDigital Library
- D. Evans and D. Larochelle. Improving security using extensible lightweight static analysis. IEEE Software, 2002. Google ScholarDigital Library
- C. Flanagan and S. N. Freund. Atomizer: A dynamic atomicity checker for multithreaded programs. In POPL, 2004. Google ScholarDigital Library
- Z. P. Fry, D. Shepherd, E. Hill, L. Pollock, and K. Vijay-Shanker. Analysing source code: Looking for useful verb-direct object pairs in all the right places. IET Software Special Issue on Natural Language in Software Development, 2008.Google ScholarCross Ref
- O. Glickman and I. Dagan. Acquiring lexical paraphrases from a single corpus. In RANLP, 2003.Google Scholar
- C. Hammer, J. Dolby, M. Vaziri, and F. Tip. Dynamic detection of atomic-set-serializability violations. In ICSE, 2008. Google ScholarDigital Library
- T. Harris and K. Fraser. Language support for lightweight transactions. SIGPLAN Not., 2003. Google ScholarDigital Library
- J. Hatcliff, Robby, and M. B. Dwyer. Verifying atomicity specifications for concurrent object-oriented software using model-checking. In VMCAI, 2004.Google ScholarCross Ref
- M. Herlihy and J. E. B. Moss. Transactional memory: Architectural support for lock-free data structures. SIGARCH Comput. Archit. News, 1993. Google ScholarDigital Library
- P. Inverardi and S. Uchitel. Proving deadlock freedom in component-based programming. In FASE, 2001. Google ScholarDigital Library
- H. Jula, D. Tralamazza, C. Zamfir, and G. Candea. Deadlock immunity: Enabling systems to defend against deadlocks. In OSDI, 2008. Google ScholarDigital Library
- D. E. Knuth. Literate programming. Computer Journal, 27(2), 1984. Google ScholarDigital Library
- D. Kramer. API documentation from source code comments: A case study of javadoc. In SIGDOC, 1999. Google ScholarDigital Library
- V. Kuznetsov, V. Chipounov, and G. Candea. Testing closed-source binary device drivers with DDT. In USENIX ATC, 2010. Google ScholarDigital Library
- Z. Lai, S.-C. Cheung, and W. K. Chan. Detecting atomic-set serializability violations in multithreaded programs through active randomized testing. In ICSE, 2010. Google ScholarDigital Library
- N. Leveson. The Therac-25 accidents.Google Scholar
- Z. Li and Y. Zhou. PR-Miner: Automatically extracting implicit programming rules and detecting violations in large software code. In FSE, 2005. Google ScholarDigital Library
- D. Lin and P. Pantel. Discovery of inference rules for question-answering. Natural Language Engineering, 2001. Google ScholarDigital Library
- B. Livshits and T. Zimmermann. DynaMine: Finding common error patterns by mining software revision histories. In FSE, 2005. Google ScholarDigital Library
- D. Lorenzoli, L. Mariani, and M. Pezzè. Automatic generation of software behavioral models. In ICSE, 2008. Google ScholarDigital Library
- S. Lu, S. Park, C. Hu, X. Ma, W. Jiang, Z. Li, R. A. Popa, and Y. Zhou. MUVI: Automatically inferring multi-variable access correlations and detecting related semantic and concurrency bugs. In SOSP, October 2007. Google ScholarDigital Library
- S. Lu, J. Tucek, F. Qin, and Y. Zhou. AVIO: Detecting atomicity violations via access interleaving invariants. In ASPLOS, 2006. Google ScholarDigital Library
- H. Malik, I. Chowdhury, H.-M. Tsou, Z. M. Jiang, and A. E. Hassan. Understanding the rationale for updating a function's comment. In ICSM, 2008.Google Scholar
- Microsoft. MSDN run-time library reference - SAL annotations. http://msdn2.microsoft.com/en-us/library/ms235402.aspx.Google Scholar
- S. Microsystems. Lock_Lint - Static data race and deadlock detection tool for C. http://developers.sun.com/sunstudio/articles/locklint.html.Google Scholar
- B. Murphy. Automating software failure reporting. Queue, 2004. Google ScholarDigital Library
- M. Musuvathi and S. Qadeer. Iterative context bounding for systematic testing of multithreaded programs. In PLDI, 2007. Google ScholarDigital Library
- M. Naik, C.-S. Park, K. Sen, and D. Gay. Effective static deadlock detection. In ICSE, 2009. Google ScholarDigital Library
- S. Narayanasamy, Z. Wang, J. Tigani, A. Edwards, and B. Calder. Automatically classifying benign and harmful data races all using replay analysis. In PLDI, 2007. Google ScholarDigital Library
- R. H. B. Netzer and B. P. Miller. Improving the accuracy of data race detection. In PPoPP, 1991. Google ScholarDigital Library
- R. O'Callahan and J.-D. Choi. Hybrid dynamic data race detection. In PPoPP, 2003. Google ScholarDigital Library
- Y. Padioleau, L. Tan, and Y. Zhou. Listening to programmers - Taxonomies and characteristics of comments in operating system code. In ICSE, May 2009. Google ScholarDigital Library
- D. Perkovic and P. J. Keleher. Online data-race detection via coherency guarantees. In OSDI, 1996. Google ScholarDigital Library
- D. Posnett, C. Bird, and P. T. Devanbu. THEX: Mining metapatterns from java. In MSR, 2010.Google ScholarCross Ref
- Z. Rakamaric. STORM: Static unit checking of concurrent programs. In ICSE Student Research Competition, 2010. Google ScholarDigital Library
- D. S. Rosenblum. Design and verification of distributed tasking supervisors for concurrent programming languages. 1988.Google Scholar
- N. Rungta, E. Mercer, and W. Visser. Efficient testing of concurrent programs with abstraction-guided symbolic execution. In SPIN, 2009. Google ScholarDigital Library
- L. Ryzhyk, P. Chubb, I. Kuz, and G. Heiser. Dingo: Taming device drivers. In EuroSys, 2009. Google ScholarDigital Library
- A. Sasturkar, R. Agarwal, L. Wang, and S. D. Stoller. Automated type-based analysis of data races and atomicity. In PPoPP, 2005. Google ScholarDigital Library
- S. Savage, M. Burrows, G. Nelson, P. Sobalvarro, and T. Anderson. Eraser: A dynamic data race detector for multithreaded programs. ACM Transactions on Computer Systems, 1997. Google ScholarDigital Library
- SecurityFocus. Software bug contributed to blackout.Google Scholar
- E. Sherman, M. B. Dwyer, and S. Elbaum. Saturation-based testing of concurrent programs. In FSE, 2009. Google ScholarDigital Library
- N. Sterling. WARLOCK - A static data race analysis tool. In USENIX Winter Technical Conference, 1993.Google Scholar
- M.-A. Storey, J. Ryall, R. I. Bull, D. Myers, and J. Singer. To do or to bug: Exploring how task annotations play a role in the work practices of software developers. In ICSE '08, 2008. Google ScholarDigital Library
- L. Tan, C. Liu, Z. Li, X. Wang, S. Lu, Y. Zhou, and C. Zhai. Bug characteristics in modern open source software. In University of Waterloo Technical Report, 2011.Google Scholar
- L. Tan, D. Yuan, G. Krishna, and Y. Zhou. /* iComment: Bugs or bad comments? */. In Proceedings of the 21st ACM Symposium on Operating Systems Principles (SOSP), 2007. Google ScholarDigital Library
- S. Thummalapenta and T. Xie. Mining exception-handling rules as sequence association rules. In ICSE, 2009. Google ScholarDigital Library
- L. Torvalds. Sparse - A semantic parser for C. http://www.kernel.org/pub/software/devel/sparse/.Google Scholar
- Y. Wang, T. Kelly, M. Kudlur, S. Lafortune, and S. Mahlke. Gadara: Dynamic deadlock avoidance for mult-threaded programs. In OSDI, 2008. Google ScholarDigital Library
- A. Wasylkowski, A. Zeller, and C. Lindig. Detecting object usage anomalies. In FSE, 2007. Google ScholarDigital Library
- M. Xu, R. Bodk, and M. D. Hill. A serializability violation detector for shared-memory server programs. In PLDI, 2005. Google ScholarDigital Library
- J. Yang, T. Kremenek, Y. Xie, and D. Engler. MECA: An extensible, expressive system and language for statically checking security properties. In CCS, 2003. Google ScholarDigital Library
- Y. Yu, T. Rodeheffer, and W. Chen. RaceTrack: Efficient detection of data race conditions via adaptive tracking. In SOSP, 2005. Google ScholarDigital Library
- W. Zhang, C. Sun, and S. Lu. ConMem: Detecting severe concurrency bugs through an effect-oriented approach. In ASPLOS, 2010. Google ScholarDigital Library
- J. Zheng, L. Williams, N. Nagappan, W. Snipes, J. P. Hudepohl, and M. A. Vouk. On the value of static analysis for fault detection in software. IEEE Trans. Softw. Eng., 32(4), 2006. Google ScholarDigital Library
- H. Zhong, L. Zhang, T. Xie, and H. Mei. Inferring resource specifications from natural language API documentation. In ASE, 2009. Google ScholarDigital Library
Index Terms
- aComment: mining annotations from comments and code to detect interrupt related concurrency bugs
Recommendations
Implementation and experimental performance evaluation of a hybrid interrupt-handling scheme
The performance of network hosts can be severely degraded when subjected to heavy traffic of today's Gigabit networks. This degradation occurs as a result of the interrupt overhead associated with the high rate of packet arrivals. NAPI, a packet ...
TxRace: Efficient Data Race Detection Using Commodity Hardware Transactional Memory
ASPLOS '16Detecting data races is important for debugging shared-memory multithreaded programs, but the high runtime overhead prevents the wide use of dynamic data race detectors. This paper presents TxRace, a new software data race detector that leverages ...
The Kaya OS project and the μMPS hardware emulator
Ideally, the most meaningful learning experience for students in an undergraduate OS course would be to develop fully-functional OS's on their own. This can be accomplished using μmps, a hardware emulator for a pedagogically undergraduate-appropriate ...
Comments