research-article

Reducing performance non-determinism via cache-aware page allocation strategies

Authors:

Tomas KaliberaAuthors Info & Claims

WOSP/SIPEW '10: Proceedings of the first joint WOSP/SIPEW international conference on Performance engineering

Pages 223 - 234

https://doi.org/10.1145/1712605.1712640

Published: 28 January 2010 Publication History

Abstract

Performance non-determinism in computer systems complicates evaluation, use, and even development of these systems. In performance evaluation via benchmarking and simulation, non-determinism requires long executions and more complex experiment design. Real-time systems are hard to dimension and tune with non-determinism. The slower benchmarking also slows down system development, as it takes developers longer to see performance implications of their modifications.

Cache-unaware physical page allocation in an operating system is believed to be a significant cause of non-determinism, but there is no published empirical study that would confirm it.

We provide such a study for the Linux operating system, comparing the default cache-unaware page allocation strategy to known cache-aware strategies, page coloring and bin hopping. We have implemented a framework for page allocation strategies in the Linux kernel, employed it for these two strategies, and measured the non-determinism on a large and diverse set of benchmarks. We propose a statistical technique which allows to classify different kinds of performance non-determinism and evaluate their magnitudes. Application of our technique reveals that the two strategies do reduce performance non-determinism without significantly increasing mean response time.

References

[1]

Andy Georges, Lieven Eeckhout, and Dries Buytaert. Java performance evaluation through rigorous replay compilation. In OOPSLA '08: Proceedings of the 23rd ACM SIGPLAN conference on Object-oriented programming systems languages and applications, pages 367--384, New York, NY, USA, 2008. ACM.

Digital Library

[2]

Alaa R. Alameldeen and David A. Wood. Variability in architectural simulations of multi-threaded workloads. In HPCA, pages 7--18, 2003.

Digital Library

[3]

Dayong Gu, Clark Verbrugge, and Etienne Gagnon. Code layout as a source of noise in JVM performance. In Component And Middleware Performance Workshop, OOPSLA 2004, October 2004.

[4]

Roldan Pozo and Bruce Miller. SciMark 2.0 benchmark. http://math.nist.gov/scimark2/, 2005.

[5]

Philippe Gerum. Xenomai -- implementing a RTOS emulation framework on GNU/Linux. http://http://www.xenomai.org/documentation/branches/v2.4.x/pdf/xenomai.pdf, 2004.

[6]

Roberto Bucher and Lorenzo Dozio. CACSD under RTAI Linux with RTAI-LAB. In Realtime Linux Workshop, 2003.

[7]

David F. Bacon, Perry Chang, and V.T. Rajan. A real-time garbage collector with low overhead and consistent utilization. pages 285--298, New Orleans, Louisiana, January 2003.

Digital Library

[8]

Distributed Systems Research Group. Mono regression benchmarking. http://dsrg.mff.cuni.cz/projects/mono, 2005.

[9]

Tomas Kalibera, Lubomir Bulej, and Petr Tuma. Benchmark precision and random initial state. In Proceedings of SPECTS 2005, pages 853--862, San Diego, CA, USA, July 2005. SCS.

[10]

R.E. Kessler and Mark D. Hill. Page placement algorithms for large real-indexed caches. ACM Trans. Comput. Syst., 10(4):338--359, 1992.

Digital Library

[11]

Richard McDougall and Jim Mauro. Solaris Internals: Solaris 10 and OpenSolaris Kernel Architecture. Sun Microsystems Press, Prentice Hall, Aug 2006.

[12]

Mark E. Russinovich and David A. Solomon. Microsoft Windows Internals, Fourth Edition: Microsoft Windows Server(TM) 2003, Windows XP, and Windows 2000 (Pro-Developer). Microsoft Press, Redmond, WA, USA, 2004.

Digital Library

[13]

Matthew Dillon. Design elements of the FreeBSD VM system. http://www.freebsd.org/doc/en/articles/vm-design, 2001.

[14]

Edouard Bugnion, Jennifer M. Anderson, Todd C. Mowry, Mendel Rosenblum, and Monica S. Lam. Compiler-directed page coloring for multiprocessors. In ASPLOS-VII: Proceedings of the seventh international conference on Architectural support for programming languages and operating systems, pages 244--255, New York, NY, USA, 1996. ACM Press.

Digital Library

[15]

Linus Torvalds. Linux kernel mailing list: Page coloring. http://lkml.org/lkml/2003/12/27/81, Dec 2003.

[16]

Richard Gooch. Linux kernel patches I've written, patches for the 2.2.x series kernel. http://www.atnf.csiro.au/people/rgooch/linux/kernel-patches.html, 1999.

[17]

Jason Papadopoulos. Linux kernel mailing list: Rewritten page coloring for 2.4.20 kernel. http://lkml.org/lkml/2003/1/4/221, Jan 2003.

[18]

Michal Hocko. Tuning virtual memory for performance. http://dsrg.mff.cuni.cz/projects/vmtune, 2007.

[19]

Tomas Kalibera and Petr Tuma. Precise regression benchmarking with random effects: Improving Mono benchmark results. In Andras Horvath and Miklos Telek, editors, Formal Methods and Stochastic Models for Performance Evaluation, volume 4054 of Lecture Notes in Computer Science, pages 63--77. Springer, June 2006.

Digital Library

[20]

Erik G. Hallnor and Steven K. Reinhardt. A fully associative software-managed cache design. SIGARCH Comput. Archit. News, 28(2):107--116, 2000.

Digital Library

[21]

Mel Gorman. Understanding the Linux Virtual Memory Manager. Prentice Hall PTR, Upper Saddle River, NJ, USA, 2004.

Digital Library

[22]

Michal Hocko. Tuning virtual memory for performance. Master's thesis, Faculty of Mathematics and Physics, Charles University, Prague, 2007. http://dsrg.mff.cuni.cz/publications/Hocko-master.pdf.

[23]

Stephane Eranian. Perfmon2: a flexible performance monitoring interface for Linux. In Proceedings of the Linux Symposium, pages 269--287, 2006. http://www.linuxsymposium.org/2006/proceedings.php.

[24]

Stephane Eranian et. al. Perfmon2: the hardware-based performance monitoring interface for Linux. http://perfmon2.sourceforge.net/, 2007.

[25]

Raj Jain. The Art of Computer Systems Performance Analysis: Techniques for Experimental Design, Measurement, Simulation, and Modeling. Wiley-Interscience, New York, NY, USA, April 1991.

[26]

David J. Lilja. Measuring Computer Performance: A Practitioner's Guide. Cambridge University Press, 2000.

Digital Library

[27]

Larry Wasserman. All of Statistics: A Concise Course in Statistical Inference. Springer, New York, NY, USA, September 2004.

Digital Library

[28]

Charles E. McCulloch and Shayle R. Searle. Generalized, Linear and Mixed Models. Wiley-Interscience, New York, NY, USA, 2001.

[29]

Tomas Kalibera, Lubomir Bulej, and Petr Tuma. Automated detection of performance regressions: The Mono experience. In MASCOTS, pages 183--190. IEEE Computer Society, 2005.

Digital Library

[30]

Satyendra Bahadur, Viswanathan Kalyanakrishnan, and James Westall. An empirical study of the effects of careful page placement in Linux. In ACM-SE 36: Proceedings of the 36th annual Southeast regional conference, pages 241--250, New York, NY, USA, 1998. ACM Press.

Digital Library

[31]

Don Mayer and O. Buneman. FFT benchmark. ftp://ftp.nosc.mil/pub/aburto/fft.

[32]

Novell, Inc. The Mono Project. http://www.mono-project.com, 2006.

[33]

Chris Re and Werner Vogels. SciMark -- C#. http://rotor.cs.cornell.edu/SciMark/, 2004.

[34]

Árpád Beszédes, Rudolf Ferenc, Tamás Gergely, Tibor Gyimóthy, Gábor Lóki, and László Vidács. Csibe benchmark: One year perspective and plans. In Proceedings of the 2004 GCC Developers' Summit, pages 7--15, June 2004.

Cited By

Chen YTian YZhou RMartínez Castro DGuo DZhou Q(2024)NDSTRNG: Non-Deterministic Sampling-Based True Random Number Generator on SoC FPGA SystemsIEEE Transactions on Computers10.1109/TC.2024.336595573:5(1313-1326)Online publication date: 14-Feb-2024
https://dl.acm.org/doi/10.1109/TC.2024.3365955
Chen YZhu FTian YXu SHan LZhou QLing N(2023)LETRNG—A Lightweight and Efficient True Random Number Generator for GNU/Linux SystemsTsinghua Science and Technology10.26599/TST.2022.901000528:2(370-385)Online publication date: Apr-2023
https://doi.org/10.26599/TST.2022.9010005
Gosakan KHan JKuszmaul WMubarek IMukherjee NSriram KTagliavini GWest EBender MBhattacharjee AConway AFarach-Colton MGandhi JJohnson RKannan SPorter DAamodt TJerger NSwift M(2023)Mosaic Pages: Big TLB Reach with Small PagesProceedings of the 28th ACM International Conference on Architectural Support for Programming Languages and Operating Systems, Volume 310.1145/3582016.3582021(433-448)Online publication date: 25-Mar-2023
https://dl.acm.org/doi/10.1145/3582016.3582021
Show More Cited By

Index Terms

Reducing performance non-determinism via cache-aware page allocation strategies

Recommendations

Prioritizing versions for performance regression testing: The Pharo case
Highlights
- An empirical study of real-world performance variations.
- A sampling technique ...
Abstract Context
Software performance may suffer regressions caused by source code changes. Measuring performance at each new software version is useful for early detection of performance regressions. However, systematically running ...
Page coloring synchronization for improving cache performance in virtualization environment
ICCSA'11: Proceedings of the 2011 international conference on Computational science and its applications - Volume Part III

The paging scheme randomly translates the virtual address into the physical address. Thus, it can lead to some serious problems like performance non-determinism and poor cache performance. In order to resolve these problems, page coloring is applied to ...
Performance regression detection in DevOps
ICSE '20: Proceedings of the ACM/IEEE 42nd International Conference on Software Engineering: Companion Proceedings

Performance is an important aspect of software quality. The goals of performance are typically defined by setting upper and lower bounds for response time and throughput of a system and physical level measurements such as CPU, memory and I/O. To meet ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

WOSP/SIPEW '10: Proceedings of the first joint WOSP/SIPEW international conference on Performance engineering

January 2010

294 pages

ISBN:9781605585635

DOI:10.1145/1712605

General Chairs:
Alan Adamson
SPEC Board of Directors
,
André Bondi
Siemens Corporate Research, USA
,
Program Chairs:
Carlos Juiz
University of the Balearic Islands, Spain
,
Mark S. Squillante
IBM T.J. Watson Research Center, USA

Copyright © 2010 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 28 January 2010

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

WOSP/SIPEW'10

Sponsor:

WOSP/SIPEW'10: First Joint WOSP/SIPEW International Conference on Performance Engineering

January 28 - 30, 2010

California, San Jose, USA

Acceptance Rates

Overall Acceptance Rate 149 of 241 submissions, 62%

Upcoming Conference

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

11
Total Citations
View Citations
225
Total Downloads

Downloads (Last 12 months)2
Downloads (Last 6 weeks)0

Reflects downloads up to 05 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Chen YTian YZhou RMartínez Castro DGuo DZhou Q(2024)NDSTRNG: Non-Deterministic Sampling-Based True Random Number Generator on SoC FPGA SystemsIEEE Transactions on Computers10.1109/TC.2024.336595573:5(1313-1326)Online publication date: 14-Feb-2024
https://dl.acm.org/doi/10.1109/TC.2024.3365955
Chen YZhu FTian YXu SHan LZhou QLing N(2023)LETRNG—A Lightweight and Efficient True Random Number Generator for GNU/Linux SystemsTsinghua Science and Technology10.26599/TST.2022.901000528:2(370-385)Online publication date: Apr-2023
https://doi.org/10.26599/TST.2022.9010005
Gosakan KHan JKuszmaul WMubarek IMukherjee NSriram KTagliavini GWest EBender MBhattacharjee AConway AFarach-Colton MGandhi JJohnson RKannan SPorter DAamodt TJerger NSwift M(2023)Mosaic Pages: Big TLB Reach with Small PagesProceedings of the 28th ACM International Conference on Architectural Support for Programming Languages and Operating Systems, Volume 310.1145/3582016.3582021(433-448)Online publication date: 25-Mar-2023
https://dl.acm.org/doi/10.1145/3582016.3582021
Shang XJia WShan JDing XBorcea C(2023)Reestablishing Page Placement Mechanisms for Nested VirtualizationIEEE Transactions on Cloud Computing10.1109/TCC.2023.3276368(1-12)Online publication date: 2023
https://doi.org/10.1109/TCC.2023.3276368
Bruel PMittal VMilojicic DFaloutsos MFrachtenberg E(2023)Revisiting Performance Evaluation in the Age of Uncertainty2023 IEEE 30th International Conference on High Performance Computing, Data and Analytics Workshop (HiPCW)10.1109/HiPCW61695.2023.00012(23-30)Online publication date: 18-Dec-2023
https://doi.org/10.1109/HiPCW61695.2023.00012
Mururu GRavichandran KGavrilovska APande S(2020)Generating Robust Parallel Programs via Model Driven Prediction of Compiler Optimizations for Non-determinismProceedings of the 49th International Conference on Parallel Processing10.1145/3404397.3404464(1-12)Online publication date: 17-Aug-2020
https://dl.acm.org/doi/10.1145/3404397.3404464
Suresh AGandhi A(2019)Using Variability as a Guiding Principle to Reduce Latency in Web Applications via OS ProfilingThe World Wide Web Conference10.1145/3308558.3313406(1759-1770)Online publication date: 13-May-2019
https://dl.acm.org/doi/10.1145/3308558.3313406
Beyer DLöwe SWendler P(2017)Reliable benchmarking: requirements and solutionsInternational Journal on Software Tools for Technology Transfer10.1007/s10009-017-0469-yOnline publication date: 3-Nov-2017
https://doi.org/10.1007/s10009-017-0469-y
Vitek JKalibera T(2012)R3ACM SIGPLAN Notices10.1145/2442776.244278147:4a(30-36)Online publication date: 18-Mar-2012
https://dl.acm.org/doi/10.1145/2442776.2442781
Pusukuri KGupta RBhuyan L(2012)Thread TranquilizerACM Transactions on Architecture and Code Optimization10.1145/2086696.20867258:4(1-21)Online publication date: 26-Jan-2012
https://dl.acm.org/doi/10.1145/2086696.2086725
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten