skip to main content
10.1145/1950365.1950401acmconferencesArticle/Chapter ViewAbstractPublication PagesasplosConference Proceedingsconference-collections
research-article

Faults in linux: ten years later

Published: 05 March 2011 Publication History

Abstract

In 2001, Chou et al. published a study of faults found by applying a static analyzer to Linux versions 1.0 through 2.4.1. A major result of their work was that the drivers directory contained up to 7 times more of certain kinds of faults than other directories. This result inspired a number of development and research efforts on improving the reliability of driver code. Today Linux is used in a much wider range of environments, provides a much wider range of services, and has adopted a new development and release model. What has been the impact of these changes on code quality? Are drivers still a major problem?
To answer these questions, we have transported the experiments of Chou et al. to Linux versions 2.6.0 to 2.6.33, released between late 2003 and early 2010. We find that Linux has more than doubled in size during this period, but that the number of faults per line of code has been decreasing. And, even though drivers still accounts for a large part of the kernel code and contains the most faults, its fault rate is now below that of other directories, such as arch (HAL) and fs (file systems). These results can guide further development and research efforts. To enable others to continually update these results as Linux evolves, we define our experimental protocol and make our checkers and results available in a public archive.

References

[1]
A. Aiken, S. Bugrara, I. Dillig, T. Dillig, B. Hackett, and P. Hawkins. An overview of the Saturn project. In Proceedings of the 7th ACM SIGPLAN-SIGSOFT Workshop on Program Analysis for Software Tools and Engineering, PASTE'07, pages 43--48, San Diego, CA, June 2007.
[2]
J. Brunel, D. Doligez, R. R. Hansen, J. Lawall, and G. Muller. A foundation for flow-based program matching using temporal logic and model checking. In The 36th annual ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages, pages 114--126, Savannah, GA, USA, Jan. 2009.
[3]
How is the department of homeland security involved?, 2009. http://scan.coverity.com/faq.html#how-department-homeland-security-in%volved.
[4]
Checkpatch. http://www.codemonkey.org.uk/projects/checkpatch/.
[5]
A. Chou, J. Yang, B. Chelf, S. Hallem, and D. Engler. An empirical study of operating systems errors. In Proceedings of the 18th ACM Symposium on Operating System Principles, pages 73--88, Banff, Canada, Oct. 2001.
[6]
Comedi: Linux Control and Mesurement Device Interface. http://www.comedi.org/.
[7]
J. Corbet. The age of kernel code in various subsystems, Feb. 2010. http://lwn.net/Articles/374622/.
[8]
J. Corbet. How old is our kernel?, Feb. 2010. http://lwn.net/Articles/374574/.
[9]
Static source code analysis, static analysis, software quality tools by Coverity Inc. http://www.coverity.com/, 2008.
[10]
A. Depoutovitch and M. Stumm. Otherworld -- giving applications a chance to survive OS kernel crashes. In ACM EuroSys, pages 181--194, Paris, France, Apr. 2010.
[11]
Fedora project, 2010. http://fedoraproject.org/.
[12]
J. N. Herder, H. Bos, B. Gras, P. Homburg, and A. S. Tanenbaum. Fault isolation for device drivers. In 2009 IEEE/IFIP International Conference on Dependable Systems and Networks (DSN), pages 33--42, Estoril, Portugal, June 2009.
[13]
IEEE std 982.2--1988 IEEE guide for the use of IEEE standard dictionary of measures to produce reliable software, 1988.
[14]
A. Israeli and D. G. Feitelson. The Linux kernel as a case study in software evolution. Journal of Systems and Software, 83(3):485--501, 2010.
[15]
J. L. Lawall, J. Brunel, R. R. Hansen, H. Stuart, G. Muller, and N. Palix. WYSIWIB: A declarative approach to finding protocols and bugs in Linux code. In The 39th Annual IEEE/IFIP International Conference on Dependable Systems and Networks, pages 43--52, Estoril, Portugal, June 2009.
[16]
Z. Li and Y. Zhou. PR-Miner: automatically extracting implicit programming rules and detecting violations in large software code. In Proceedings of the 10th European Software Engineering Conference held jointly with 13th ACM SIGSOFT International Symposium on Foundations of Software Engineering, pages 306--315, Lisbon, Portugal, Sept. 2005.
[17]
Lkml: The Linux kernel mailing list. http://lkml.org/.
[18]
S. Lu, S. Park, E. Seo, and Y. Zhou. Learning from mistakes: a comprehensive study on real world concurrency bug characteristics. In Architectural Support for Programming Languages and Operating Systems (ASPLOS), pages 329--339, Seattle, WA, USA, Mar. 2008.
[19]
T. J. McCabe. A complexity measure. IEEE Transactions on Software Engineering, 2(4):308--320, July 1976.
[20]
P. E. McKenney and J. Walpole. Introducing technology into the Linux kernel: a case study. ACM SIGOPS Operating Systems Review, 42(5):4--17, 2008.
[21]
J. C. Munson and S. G. Elbaum. Code churn: A measure for estimating the impact of code change. In International Conference Software Maintenance (ICSM), pages 24--31, 1998.
[22]
N. Nagappan and T. Ball. Use of relative code churn measures to predict system defect density. In 27th International Conference on Software Engineering (ICSE), pages 284--292, St. Louis, Missouri, USA, May 2005.
[23]
P. Oman and J. Hagemeister. Construction and testing of polynomials predicting software maintainability. Journal of Systems and Software, 24(3):251--266, 1994.
[24]
Y. Padioleau, J. Lawall, R. R. Hansen, and G. Muller. Documenting and automating collateral evolutions in Linux device drivers. In EuroSys 2008, pages 247--260, Glasgow, Scotland, Mar. 2008.
[25]
N. Palix, J. Lawall, and G. Muller. Tracking code patterns over multiple software versions with Herodotos. In Proc. of the ACM International Conference on Aspect-Oriented Software Development, AOSD'10, pages 169--180, Rennes and Saint Malo, France, Mar. 2010.
[26]
N. Palix, J. L. Lawall, and G. Muller. Herodotos: A tool to expose bugs' lives. Research report RR-6984, INRIA, July 2009.
[27]
N. Palix, S. Saha, G. Thomas, C. Calvès, J. Lawall, and G. Muller. Database of Faults in Linux: Ten Years Later, Aug. 2010. http://hal.inria.fr/docs/00/50/92/56/ANNEX/10years.sql.pg_dump.
[28]
N. Palix, S. Saha, G. Thomas, C. Calvès, J. Lawall, and G. Muller. Faults in Linux: Ten years later. Research report RR-7357, INRIA, Aug. 2010.
[29]
N. Palix, S. Saha, G. Thomas, C. Calvès, J. Lawall, and G. Muller. Website of Faults in Linux: Ten Years Later, Dec. 2010. http://faultlinux.lip6.fr/.
[30]
E. S. Raymond. The Cathedral and the Bazaar: Musings on Linux and Open Source by an Accidental Revolutionary. O'Reilly & Associates, Inc., 2001.
[31]
D. Searls. Sparse, Linus & the Lunatics, Nov. 2004. Available at http://www.linuxjournal.com/article/7272.
[32]
X. Song, H. Chen, and B. Zang. Why software hangs and what can be done with it. In International Conference on Dependable Systems and Networks (DSN 2010), Chicago, IL, USA, June 2010.
[33]
Sparse. https://sparse.wiki.kernel.org/index.php/Main_Page.
[34]
B. Spencer. Local kernel exploit in /dev/net/tun. http://grsecurity.net/ spender/cheddar_bay.tgz.
[35]
M. M. Swift, M. Annamalai, B. N. Bershad, and H. M. Levy. Recovering device drivers. ACM Transactions on Computer Systems, 24(4):333--360, 2006.
[36]
The Kernel Janitors. Smatch, the source matcher, 2010. Available at http://smatch.sourceforge.net.
[37]
Ubuntu, 2010. http://www.ubuntu.com/.
[38]
D. Wheeler. Flawfinder home page. Web page: http://www.dwheeler.com/flawfinder/, Oct. 2006.
[39]
D. A. Wheeler. SLOCCount. http://www.dwheeler.com/sloccount/.

Cited By

View all
  • (2024)Balancing analysis time and bug detectionProceedings of the 2024 USENIX Conference on Usenix Annual Technical Conference10.5555/3691992.3692023(493-508)Online publication date: 10-Jul-2024
  • (2024)Cloud-Native Database Systems and Unikernels: Reimagining OS Abstractions for Modern HardwareProceedings of the VLDB Endowment10.14778/3659437.365946217:8(2115-2122)Online publication date: 1-Apr-2024
  • (2024)Veld: Verified Linux DriversProceedings of the 2nd Workshop on Kernel Isolation, Safety and Verification10.1145/3698576.3698766(23-30)Online publication date: 4-Nov-2024
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
ASPLOS XVI: Proceedings of the sixteenth international conference on Architectural support for programming languages and operating systems
March 2011
432 pages
ISBN:9781450302661
DOI:10.1145/1950365
  • cover image ACM SIGPLAN Notices
    ACM SIGPLAN Notices  Volume 46, Issue 3
    ASPLOS '11
    March 2011
    407 pages
    ISSN:0362-1340
    EISSN:1558-1160
    DOI:10.1145/1961296
    Issue’s Table of Contents
  • cover image ACM SIGARCH Computer Architecture News
    ACM SIGARCH Computer Architecture News  Volume 39, Issue 1
    ASPLOS '11
    March 2011
    407 pages
    ISSN:0163-5964
    DOI:10.1145/1961295
    Issue’s Table of Contents
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 05 March 2011

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. fault-finding tools
  2. linux

Qualifiers

  • Research-article

Conference

ASPLOS'11

Acceptance Rates

Overall Acceptance Rate 535 of 2,713 submissions, 20%

Upcoming Conference

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)101
  • Downloads (Last 6 weeks)9
Reflects downloads up to 15 Jan 2025

Other Metrics

Citations

Cited By

View all
  • (2024)Balancing analysis time and bug detectionProceedings of the 2024 USENIX Conference on Usenix Annual Technical Conference10.5555/3691992.3692023(493-508)Online publication date: 10-Jul-2024
  • (2024)Cloud-Native Database Systems and Unikernels: Reimagining OS Abstractions for Modern HardwareProceedings of the VLDB Endowment10.14778/3659437.365946217:8(2115-2122)Online publication date: 1-Apr-2024
  • (2024)Veld: Verified Linux DriversProceedings of the 2nd Workshop on Kernel Isolation, Safety and Verification10.1145/3698576.3698766(23-30)Online publication date: 4-Nov-2024
  • (2024)Reboot-Based Recovery of Unikernels at the Component Level2024 54th Annual IEEE/IFIP International Conference on Dependable Systems and Networks (DSN)10.1109/DSN58291.2024.00017(15-28)Online publication date: 24-Jun-2024
  • (2024)eBPF: Pioneering Kernel Programmability and System Observability - Past, Present, and Future Insights2024 3rd International Conference on Artificial Intelligence and Computer Information Technology (AICIT)10.1109/AICIT62434.2024.10730620(1-10)Online publication date: 20-Sep-2024
  • (2023)Takeaways of Implementing a Native Rust UDP Tunneling Network Driver in the Linux KernelProceedings of the 12th Workshop on Programming Languages and Operating Systems10.1145/3623759.3624547(18-25)Online publication date: 23-Oct-2023
  • (2023)Exploring the Impact of Code Clones on Deep Learning SoftwareACM Transactions on Software Engineering and Methodology10.1145/360718132:6(1-34)Online publication date: 3-Jul-2023
  • (2023)Minimizing a Smartphone's TCB for Security-Critical Programs with Exclusively-Used, Physically-Isolated, Statically-Partitioned HardwareProceedings of the 21st Annual International Conference on Mobile Systems, Applications and Services10.1145/3581791.3596864(233-246)Online publication date: 18-Jun-2023
  • (2023)iBiR: Bug-report-driven Fault InjectionACM Transactions on Software Engineering and Methodology10.1145/354294632:2(1-31)Online publication date: 30-Mar-2023
  • (2023)Understanding and Mitigating Twin Function Misuses in Operating System KernelIEEE Transactions on Computers10.1109/TC.2023.324036572:8(2181-2193)Online publication date: 1-Aug-2023
  • Show More Cited By

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Media

Figures

Other

Tables

Share

Share

Share this Publication link

Share on social media