Article

Fine grained kernel logging with KLogger: experience and insights

Authors:

Scott Kirkpatrick,

Dror G. FeitelsonAuthors Info & Claims

EuroSys '07: Proceedings of the 2nd ACM SIGOPS/EuroSys European Conference on Computer Systems 2007

Pages 259 - 272

https://doi.org/10.1145/1272996.1273023

Published: 21 March 2007 Publication History

Abstract

Understanding the detailed behavior of an operating system is crucial for making informed design decisions. But such an understanding is very hard to achieve, due to the increasing complexity of such systems and the fact that they are implemented and maintained by large and diverse groups of developers. Tools like KLogger --- presented in this paper --- can help by enabling fine-grained logging of system events and the sharing of a logging infrastructure between multiple developers and researchers, facilitating a methodology where design evaluation can be an integral part of kernel development. We demonstrate the need for such methodology by a host of case studies, using KLogger to better understand various subsystems in the Linux kernel, and pinpointing overheads and problems therein.

References

[1]

J. M. Anderson, W. E. Weihl, L. M. Berc, J. Dean, S. Ghemawat, M. R. Henzinger, S.-T. A. Leung, R. L. Sites, M. T. Vandevoorde, and C. A. Waldspurger. Continuous profiling: where have all the cycles gone? In ACM Symp. on Operating Systems Principles, pages 1--14, 1997. ACM Press.

Digital Library

[2]

A. C. Arpaci-Dusseau and R. H. Arpaci-Dusseau. Information and control in gray-box systems. In ACM Symp. on Operating Systems Principles, 2001. ACM Press.

Digital Library

[3]

D. L. Black, A. Tevanian, D. B. Golub, and M. W. Young. Locking and reference counting in the Mach kernel. In Intl. Conf. on Parallel Processing, volume 2, pages 167--173, 1991.

[4]

D. P. Bovet and M. Cesati. Understanding the Linux Kernel. O'Reilly & Associates, Sebastopol, CA, USA, 2001.

Digital Library

[5]

Z. Brown. What's new in kernel developement. Linux Journal, page 10, Mar 2005.

[6]

R. Bryant, R. Forester, and J. Hawkes. Filesystem performance and scalability in Linux 2.4.17. In Usenix Annual Technical Conf. (FreeNix Track), 2002.

Digital Library

[7]

R. Bryant and J. Hawkes. Lockmeter: Highly-informative instrumentation for spin locks in the Linux kernel. In 4th Annual Linux Showcase & Conf., 2000.

Digital Library

[8]

B. M. Cantrill, M. W. Shapiro, and A. H. Leventhal. Dynamic instrumentation of production systems. In Usenix Annual Technical Conf., June 2004. The Usenix Association.

Digital Library

[9]

J. Casmira, D. Kaeli, and D. Hunter. Tracing and characterization of NT-based system workloads. Digital Tech. J., 10(1):6--21, Dec 1998.

[10]

Y. Endo and M. Seltzer. Improving interactive performance using TIPME. In Intl. Conf. on Measurement & Modeling of Computer Systems, pages 240--251, June 2000. ACM Press.

Digital Library

[11]

Y. Etsion, D. Tsafrir, and D. G. Feitelson. Effects of clock resolution on the scheduling of interactive and soft real-time processes. In Intl. Conf. on Measurement & Modeling of Computer Systems, pages 172--183, Jun 2003. ACM Press.

Digital Library

[12]

E. Frachtenberg and Y. Etsion. Hardware parallelism: Are operating systems ready? (case studies in mis-scheduling). In Workshop on the Interaction between Operating System and Computer Architecture, pages 7--15, June 2006. http://www.ideal.ece.ufl.edu/workshops/wiosca06/final.pdf.

[13]

R. Gioiosa, F. Petrini, K. Davis, and F. Lebaillif-Delamare. Analysis of system overhead on parallel computers. In IEEE Intl. Symp. on Signal Processing and Information Technology, Dec 2004. IEEE Computer Society Press.

[14]

Intel Corp. IA-32 Intel Achitecture Software Developr's Manual. Vol. 3: System Programming Guide.

[15]

M. B. Jones and J. Regehr. The problems you're having may not be the problems you think you're having: Results from a latency study of windows NT. In 7th Workshop on Hot Topics in Operating Systems, page 96, March 1999. The Usenix Association.

Digital Library

[16]

M. Kravetz and H. Franke. Enhancing the Linux scheduler. In Ottawa Linux Symp., 2001.

[17]

J. Lions. Lions' Commentary on UNIX 6th Edition. Annabooks, 1996. (reprint).

Digital Library

[18]

R. Love. Linux Kernel Development. Novell Press, Indianapolis, IN, USA, 2nd edition, 2005.

Digital Library

[19]

R. Love, L. Torvalds, A. Cox, and various kernel developers. LKML Thread: Improving interactivity. http://kerneltrap.org/node/603, Mar 2003.

[20]

J. M. Mellor-Crummey and M. L. Scott. Algorithms for scalable synchronization on shared-memory multiprocessors. ACM Trans. on Computer Systems, 9(1):21--65, Feb 1991.

Digital Library

[21]

S. W. Melvin and Y. N. Patt. The use of microcode instrumentation for development, debugging and tuning of operating system kernels. In Intl. Conf. on Measurement & Modeling of Computer Systems, pages 207--214, 1988. ACM Press.

Digital Library

[22]

J. K. Ousterhout. Why aren't operating systems getting faster as fast as hardware? In Usenix Annual Technical Conf. (Summer), Jun 1990. The Usenix Association.

[23]

P. S. Panchamukhi. Kernel debugging with Kprobes. http://www-106.ibm.com/developerworks/linux/library/l-kprobes.html, Aug 2004.

[24]

F. Petrini, D. J. Kerbyson, and S. Pakin. The case of missing supercomputer performance: Achieving optimal performance on the 8,192 processors of ASCI Q. In Supercomputing, Nov 2003.

Digital Library

[25]

M. Rosenblum, E. Bugnion, S. Devine, and S. A. Herrod. Using the simos machine simulator to study complex computer systems. ACM Trans. on Modelling & Computer Simulation, 7(1):78--103, 1997.

Digital Library

[26]

M. Rosenblum, E. Bugnion, S. A. Herrod, E. Witchel, and A. Gupta. The impact of architectural trends on operating system performance. In ACM Symp. on Operating Systems Principles, 1995. ACM Press.

Digital Library

[27]

C. Schimmel. UNIX Systems for Modern Architectures. Addison Wesley, Reading, MA, USA, 1994.

Digital Library

[28]

B. A. Schroeder. On line monitoring: A tutorial. IEEE Computer, 28(6):72--78, June 1995.

Digital Library

[29]

D. A. Solomon and M. E. Russinovich. Inside Windows 2000. Microsoft Press, Redmond, WA, USA, 3rd ed., 2000.

[30]

A. Tamches and B. P. Miller. Fine-grained dynamic instrumentation of commodity operating system kernels. In 3rd Symp. on Operating Systems Design & Impl., Feb 1999. The Usenix Association.

Digital Library

[31]

J. Torrellas, A. Gupta, and J. Hennessy. Characterizing the caching and synchronization performance of a multiprocessor operating system. In Arch. Support for Programming Languages & Operating Systems, pages 162--174, Oct. 1992. ACM Press.

Digital Library

[32]

R. Unrau, O. Krieger, B. Gamsa, and M. Stumm. Experiences with locking in a NUMA multiprocessor operating system kernel. In Symp. on Operating Systems Design & Impl., Nov 1994. The Usenix Association.

Digital Library

[33]

Various Kernel Developers. LKML Thread: My thoughts on the "new development model". http://www.lkml.org, Oct 2004.

[34]

R. W. Wisniewski and B. Rosenburg. Efficient, unified, and scalable performance monitoring for multiprocessor operating systems. In Supercomputing, Nov 2003. IEEE Computer Society.

Digital Library

[35]

K. Yaghmour and M. R. Dagenais. Measuring and characterizing system behavior using kernel-level event logging. In Usenix Annual Technical Conf., June 2000. The Usenix Association.

Digital Library

[36]

T. Zanussi, K. Yaghmour, R. W. Wisniewski, M. Dagenais, and R. Moore. An efficient unified approach for trasmitting data from kernel to user space. In Ottawa Linux Symp., Jul 2003.

Cited By

Yang XQiu KZhang Q(2024)EasyLog: An Efficient Kernel Logging Service for Machine LearningApplied Cryptography and Network Security Workshops10.1007/978-3-031-61486-6_2(16-28)Online publication date: 24-Jun-2024
https://doi.org/10.1007/978-3-031-61486-6_2
Bissyandé TRéveillère LLawall JMuller G(2016)Ahead of time static analysis for automatic generation of debugging interfaces to the Linux kernelAutomated Software Engineering10.1007/s10515-014-0152-423:1(3-41)Online publication date: 1-Mar-2016
https://dl.acm.org/doi/10.1007/s10515-014-0152-4
Larson UJonsson ELindskog S(2015)Guidance for Selecting Data Collection Mechanisms for Intrusion DetectionHandbook of Research on Emerging Developments in Data Privacy10.4018/978-1-4666-7381-6.ch016(340-370)Online publication date: 2015
https://doi.org/10.4018/978-1-4666-7381-6.ch016
Show More Cited By

Index Terms

Fine grained kernel logging with KLogger: experience and insights

Recommendations

Fine grained kernel logging with KLogger: experience and insights
EuroSys'07 Conference Proceedings

Understanding the detailed behavior of an operating system is crucial for making informed design decisions. But such an understanding is very hard to achieve, due to the increasing complexity of such systems and the fact that they are implemented and ...
Linux kernel projects for an undergraduate operating systems course
SIGCSE '10: Proceedings of the 41st ACM technical symposium on Computer science education

In this paper, we present a series of programming projects based on the Linux kernel for students in a senior-level undergraduate operating systems course. The projects we describe cover several key operating systems concepts, including process ...
Is It Possible to Automatically Port Kernel Modules?
APSys '18: Proceedings of the 9th Asia-Pacific Workshop on Systems

As essential components in Linux kernel, kernel modules (kmods) account for over 70% of Linux source code and are heavily dependent on fast evolving and non-stable kernel internal interfaces. Forward and back porting kmods to target versions of Linux ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

EuroSys '07: Proceedings of the 2nd ACM SIGOPS/EuroSys European Conference on Computer Systems 2007

March 2007

431 pages

ISBN:9781595936363

DOI:10.1145/1272996

ACM SIGOPS Operating Systems Review Volume 41, Issue 3
EuroSys'07 Conference Proceedings
June 2007
386 pages
ISSN:0163-5980
DOI:10.1145/1272998
Issue’s Table of Contents

Copyright © 2007 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGOPS: ACM Special Interest Group on Operating Systems

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 21 March 2007

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Article

Conference

EuroSys07

Sponsor:

SIGOPS

EuroSys07: Eurosys 2007 Conference

March 21 - 23, 2007

Lisbon, Portugal

Acceptance Rates

Overall Acceptance Rate 241 of 1,308 submissions, 18%

Upcoming Conference

EuroSys '25

Sponsor:
sigops

Twentieth European Conference on Computer Systems

March 30 - April 3, 2025

Rotterdam , Netherlands

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

8
Total Citations
View Citations
498
Total Downloads

Downloads (Last 12 months)6
Downloads (Last 6 weeks)1

Reflects downloads up to 05 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Yang XQiu KZhang Q(2024)EasyLog: An Efficient Kernel Logging Service for Machine LearningApplied Cryptography and Network Security Workshops10.1007/978-3-031-61486-6_2(16-28)Online publication date: 24-Jun-2024
https://doi.org/10.1007/978-3-031-61486-6_2
Bissyandé TRéveillère LLawall JMuller G(2016)Ahead of time static analysis for automatic generation of debugging interfaces to the Linux kernelAutomated Software Engineering10.1007/s10515-014-0152-423:1(3-41)Online publication date: 1-Mar-2016
https://dl.acm.org/doi/10.1007/s10515-014-0152-4
Larson UJonsson ELindskog S(2015)Guidance for Selecting Data Collection Mechanisms for Intrusion DetectionHandbook of Research on Emerging Developments in Data Privacy10.4018/978-1-4666-7381-6.ch016(340-370)Online publication date: 2015
https://doi.org/10.4018/978-1-4666-7381-6.ch016
Erlingsson ÚPeinado MPeter SBudiu MMainar-Ruiz G(2012)FayACM Transactions on Computer Systems10.1145/2382553.238255530:4(1-35)Online publication date: 1-Nov-2012
https://dl.acm.org/doi/10.1145/2382553.2382555
Erlingsson ÚPeinado MPeter SBudiu MWobber TDruschel P(2011)FayProceedings of the Twenty-Third ACM Symposium on Operating Systems Principles10.1145/2043556.2043585(311-326)Online publication date: 23-Oct-2011
https://dl.acm.org/doi/10.1145/2043556.2043585
Schöbel MPolze ALiblit BPodgurski A(2009)A runtime environment for online processing of operating system kernel eventsProceedings of the Seventh International Workshop on Dynamic Analysis10.1145/2134243.2134245(1-7)Online publication date: 20-Jul-2009
https://dl.acm.org/doi/10.1145/2134243.2134245
Tsafrir DEtsion YFeitelson D(2007)Secretly monopolizing the CPU without superuser privilegesProceedings of 16th USENIX Security Symposium on USENIX Security Symposium10.5555/1362903.1362920(1-18)Online publication date: 6-Aug-2007
https://dl.acm.org/doi/10.5555/1362903.1362920
Tsafrir D(2007)The context-switch overhead inflicted by hardware interrupts (and the enigma of do-nothing loops)Proceedings of the 2007 workshop on Experimental computer science10.1145/1281700.1281704(4-es)Online publication date: 13-Jun-2007
https://dl.acm.org/doi/10.1145/1281700.1281704

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten