skip to main content
research-article

RAPL in Action: Experiences in Using RAPL for Power Measurements

Authors Info & Claims
Published:22 March 2018Publication History
Skip Abstract Section

Abstract

To improve energy efficiency and comply with the power budgets, it is important to be able to measure the power consumption of cloud computing servers. Intel’s Running Average Power Limit (RAPL) interface is a powerful tool for this purpose. RAPL provides power limiting features and accurate energy readings for CPUs and DRAM, which are easily accessible through different interfaces on large distributed computing systems. Since its introduction, RAPL has been used extensively in power measurement and modeling. However, the advantages and disadvantages of RAPL have not been well investigated yet. To fill this gap, we conduct a series of experiments to disclose the underlying strengths and weaknesses of the RAPL interface by using both customized microbenchmarks and three well-known application level benchmarks: Stream, Stress-ng, and ParFullCMS. Moreover, to make the analysis as realistic as possible, we leverage two production-level power measurement datasets from the Taito, a supercomputing cluster of the Finnish Center of Scientific Computing and also replicate our experiments on Amazon EC2. Our results illustrate different aspects of RAPL and document the findings through comprehensive analysis. Our observations reveal that RAPL readings are highly correlated with plug power, promisingly accurate enough, and have negligible performance overhead. Experimental results suggest RAPL can be a very useful tool to measure and monitor the energy consumption of servers without deploying any complex power meters. We also show that there are still some open issues, such as driver support, non-atomicity of register updates, and unpredictable timings that might weaken the usability of RAPL in certain scenarios. For such scenarios, we pinpoint solutions and workarounds.

References

  1. VMSTAT. Retrieved from http://www.linuxcommand.org/man_pages/vmstat8.html.Google ScholarGoogle Scholar
  2. David Abdurachmanov, Peter Elmer, Giulio Eulisse, Robert Knight, Tapio Niemi, Jukka K. Nurminen, Filip Nyback, Goncalo Pestana, Zhonghong Ou, and Kashif Nizam Khan. 2014. Techniques and tools for measuring energy efficiency of scientific software applications. CoRR abs/1410.3440 (2014). Retrieved from http://arxiv.org/abs/1410.3440.Google ScholarGoogle Scholar
  3. Xi Chen, Chi Xu, Robert P. Dick, and Zhuoqing Morley Mao. 2010. Performance and power modeling in a multi-programmed multi-core environment. In Proceedings of the 47th Design Automation Conference (DAC’10). ACM, New York, NY, 813--818. Google ScholarGoogle ScholarDigital LibraryDigital Library
  4. CSC. 2017. Taito supercluster. Retrieved from https://research.csc.fi/taito-supercluster.Google ScholarGoogle Scholar
  5. Howard David, Eugene Gorbatov, Ulf R. Hanebutte, Rahul Khanna, and Christian Le. 2010. RAPL: Memory power estimation and capping. In Proceedings of the 16th ACM/IEEE International Symposium on Low Power Electronics and Design (ISLPED’10). ACM, New York, NY, 189--194. Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. Spencer Desrochers, Chad Paradis, and Vincent M. Weaver. 2016. A validation of DRAM RAPL power measurements. In Proceedings of the 2nd International Symposium on Memory Systems (MEMSYS’16). ACM, New York, NY, 455--470. Google ScholarGoogle ScholarDigital LibraryDigital Library
  7. Mohammed El Mehdi Diouri, Manuel F. Dolz, Olivier Glück, Laurent Lefèvre, Pedro Alonso, Sandra Catalán, Rafael Mayo, and Enrique S. Quintana-Ortí. 2014. Assessing power monitoring approaches for energy and power analysis of computers. Sustain. Comput.: Informat. Syst. 4, 2 (2014), 68--82.Google ScholarGoogle ScholarCross RefCross Ref
  8. Jack Dongarra, Hatem Ltaief, Piotr Luszczek, and Vincent M. Weaver. 2012. Energy footprint of advanced dense numerical linear algebra using tile algorithms on multicore architectures. In Proceedings of the 2012 2nd International Conference on Cloud and Green Computing (CGC’12). IEEE, 274--281. Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. Amazon EC2. Instance Types. Retrieved from https://aws.amazon.com/ec2/instance-types/.Google ScholarGoogle Scholar
  10. S. Agostinelli et al. 2003. Geant4- A simulation toolkit. Nucl. Instrum. Methods Phys. Res. Sec. A: Accel. Spectrom. Detect. Assoc. Equip. 506, 3 (2003), 250--303.Google ScholarGoogle ScholarCross RefCross Ref
  11. Daniel Hackenberg, Thomas Ilsche, Robert Schone, Daniel Molka, Maik Schmidt, and Wolfgang E Nagel. 2013. Power measurement techniques on standard compute nodes: A quantitative comparison. In Proceedings of the 2013 IEEE International Symposium on Performance Analysis of Systems and Software (ISPASS’13). IEEE, 194--204.Google ScholarGoogle ScholarCross RefCross Ref
  12. Daniel Hackenberg, Robert Schöne, Thomas Ilsche, Daniel Molka, Joseph Schuchart, and Robin Geyer. 2015. An energy efficiency feature survey of the intel haswell processor. In Proceedings of the International Parallel and Distributed Processing Symposium (IPDPS’15). Google ScholarGoogle ScholarDigital LibraryDigital Library
  13. D. Hackenberg, R. Schöne, T. Ilsche, D. Molka, J. Schuchart, and R. Geyer. 2015. An energy efficiency feature survey of the intel haswell processor. In Proceedings of the 2015 IEEE International Parallel and Distributed Processing Symposium Workshop. 896--904. Google ScholarGoogle ScholarDigital LibraryDigital Library
  14. Marcus Hähnel, Björn Döbel, Marcus Völp, and Hermann Härtig. 2012. Measuring energy consumption for short code paths using RAPL. ACM SIGMETRICS Perform. Eval. Rev. 40, 3 (2012), 13--17. Google ScholarGoogle ScholarDigital LibraryDigital Library
  15. Mikael Hirki. 2015. Energy and Performance Profiling of Scientific Computing. Master’s thesis. Aalto University.Google ScholarGoogle Scholar
  16. Mikael Hirki. 2017. RAPL Testing and Instruction Decoder Benchmarks. Retrieved from https://github.com/mhirki/idq-bench2.Google ScholarGoogle Scholar
  17. Mikael Hirki, Zhonghong Ou, Kashif Nizam Khan, Jukka K Nurminen, and Tapio Niemi. 2016. Empirical study of the power consumption of the x86-64 instruction decoder. In Proceedings of the USENIX Workshop on Cool Topics on Sustainable Data Centers (CoolDC’16). USENIX Association.Google ScholarGoogle Scholar
  18. Song Huang, Michael Lang, Scott Pakin, and Song Fu. 2015. Measurement and characterization of haswell power and energy consumption. In Proceedings of the 3rd International Workshop on Energy Efficient Supercomputing (E2SC’15). ACM, New York, NY. Google ScholarGoogle ScholarDigital LibraryDigital Library
  19. T. Ilsche, D. Hackenberg, S. Graul, R. Schöne, and J. Schuchart. 2015. Power measurements for compute nodes: Improving sampling rates, granularity and accuracy. In Proceedings of the 2015 6th International Green and Sustainable Computing Conference (IGSC’15). 1--8. Google ScholarGoogle ScholarDigital LibraryDigital Library
  20. Intel Corporation 2015. Intel® 64 and IA-32 Architectures Software Developer’s Manual, Volume 3, System Programming Guide. Intel Corporation.Google ScholarGoogle Scholar
  21. R. Kavanagh, D. Armstrong, and K. Djemame. 2016. Accuracy of energy model calibration with IPMI. In Proceedings of the 2016 IEEE 9th International Conference on Cloud Computing (CLOUD’16). 648--655.Google ScholarGoogle Scholar
  22. Jaimie Kelley, Christopher Stewart, Devesh Tiwari, and Saurabh Gupta. 2016. Adaptive power profiling for many-core HPC architectures. In Proceedings of the International Conference on Autonomic Computing.Google ScholarGoogle ScholarCross RefCross Ref
  23. K. N. Khan, F. Nybäck, Z. Ou, J. K. Nurminen, T. Niemi, G. Eulisse, P. Elmer, and D. Abdurachmanov. 2015. Energy profiling using IgProf. In Proceedings of the 2015 15th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing. 1115--1118.Google ScholarGoogle Scholar
  24. Kashif Nizam Khan, Zhonghong Ou, Mikael Hirki, Jukka K. Nurminen, and Tapio Niemi. 2016. How much power does your server consume? Estimating wall socket power using RAPL measurements. Comput. Sci. Res. Dev. 31, 4 (2016), 207--214. Google ScholarGoogle ScholarDigital LibraryDigital Library
  25. R. Khanna, F. Zuhayri, M. Nachimuthu, C. Le, and M. J. Kumar. 2011. Unified extensible firmware interface: An innovative approach to DRAM power control. In Proceedings of the 2011 International Conference on Energy Aware Computing. 1--6.Google ScholarGoogle Scholar
  26. Gary Lawson, Masha Sosonkina, and Yuzhong Shen. 2015. Towards modeling energy consumption of Xeon Phi. arXiv:1505.06539 (2015).Google ScholarGoogle Scholar
  27. H. Liu. 2011. A measurement study of server utilization in public clouds. In Proceedings of the 2011 IEEE 9th International Conference on Dependable, Autonomic and Secure Computing. 435--442. Google ScholarGoogle ScholarDigital LibraryDigital Library
  28. Ioannis Manousakis, Foivos S. Zakkak, Polyvios Pratikakis, and Dimitrios S. Nikolopoulos. 2015. TProf: An energy profiler for task-parallel programs. Sustainable Computing: Informatics and Systems 5 (2015), 1--13. http://www.sciencedirect.com/science/article/pii/S2210537914000390.Google ScholarGoogle ScholarCross RefCross Ref
  29. John D. McCalpin. 1995. Memory bandwidth and machine balance in current high performance computers. IEEE Comput. Soc. Tech. Committee Comput. Architect. (TCCA) Newsletter (Dec. 1995), 19--25.Google ScholarGoogle Scholar
  30. John C. McCullough, Yuvraj Agarwal, Jaideep Chandrashekar, Sathyanarayan Kuppuswamy, Alex C. Snoeren, and Rajesh K. Gupta. 2011. Evaluating the effectiveness of model-based power characterization. In Proceedings of the 2011 USENIX Conference on USENIX Annual Technical Conference (USENIXATC’11). USENIX Association, Berkeley, CA, 12--12. Retrieved from DOI: http://dl.acm.org/citation.cfm?id=2002181.2002193. Google ScholarGoogle ScholarDigital LibraryDigital Library
  31. C. Möbius, W. Dargie, and A. Schill. 2014. Power consumption estimation models for processors, virtual machines, and servers. IEEE Trans. Parallel Distrib. Syst. 25, 6 (June 2014), 1600--1614. Google ScholarGoogle ScholarDigital LibraryDigital Library
  32. Chad M. Paradis. 2015. Detailed Low-cost Energy and Power Monitoring of Computing Systems. Master’s thesis. University of Maine.Google ScholarGoogle Scholar
  33. Tapasya Patki, David K. Lowenthal, Barry Rountree, Martin Schulz, and Bronis R. de Supinski. 2013. Exploring hardware overprovisioning in power-constrained, high performance computing. In Proceedings of the 27th International ACM Conference on International Conference on Supercomputing (ICS’13). ACM, New York, NY, 173--182. Google ScholarGoogle ScholarDigital LibraryDigital Library
  34. Plugwise. 2017. Energy management systems of the 21st century. Retrieved from https://www.plugwise.com/.Google ScholarGoogle Scholar
  35. Harald Servat, Germán Llort, Judit Giménez, and Jesús Labarta. 2016. Detailed and simultaneous power and performance analysis. Concurrency and Computation: Practice and Experience 28, 2 (2016), 252--273. Google ScholarGoogle ScholarDigital LibraryDigital Library
  36. A. Skrenes and C. Williamson. 2016. Experimental calibration and validation of a speed scaling simulator. In Proceedings of the 2016 IEEE 24th International Symposium on Modeling, Analysis and Simulation of Computer and Telecommunication Systems (MASCOTS’16). 105--114.Google ScholarGoogle Scholar
  37. Balaji Subramaniam and Wu-chun Feng. 2015. On the energy proportionality of scale-out workloads. CoRR abs/1501.02729 (2015). Retrieved from DOI: http://arxiv.org/abs/1501.02729.Google ScholarGoogle Scholar
  38. Balaji Subramaniam and Wu-chun Feng. 2013. Towards energy-proportional computing for enterprise-class server workloads. In Proceedings of the 4th ACM/SPEC International Conference on Performance Engineering (ICPE’13). ACM, New York, NY, 15--26. Google ScholarGoogle ScholarDigital LibraryDigital Library
  39. Vincent Weaver. 2015. rapl-read.c. Retrieved from http://web.eece.maine.edu/ vweaver/projects/rapl/rapl-read.c.Google ScholarGoogle Scholar
  40. V. M. Weaver, M. Johnson, K. Kasichayanula, J. Ralph, P. Luszczek, D. Terpstra, and S. Moore. 2012. Measuring energy and power with PAPI. In Proceedings of the 2012 41st International Conference on Parallel Processing Workshops. 262--268. Google ScholarGoogle ScholarDigital LibraryDigital Library
  41. Vincent M. Weaver, Matt Johnson, Kiran Kasichayanula, James Ralph, Piotr Luszczek, Dan Terpstra, and Shirley Moore. 2012. Measuring energy and power with PAPI. In Proceedings of the 2012 41st International Conference on Parallel Processing Workshops (ICPPW’12). IEEE Computer Society, Washington, DC, 262--268. Google ScholarGoogle ScholarDigital LibraryDigital Library
  42. Simon N. Wood. 2017. Generalized Additive Models: An Introduction with R. CRC Press.Google ScholarGoogle ScholarDigital LibraryDigital Library
  43. Yan Zhai, Xiao Zhang, Stephane Eranian, Lingjia Tang, and Jason Mars. 2014. HaPPy: Hyperthread-aware power profiling dynamically. In Proceedings of the 2014 USENIX Annual Technical Conference (USENIXATC’14). USENIX Association, Philadelphia, PA, 211--217. Google ScholarGoogle ScholarDigital LibraryDigital Library
  44. Huazhe Zhang and Henry Hoffmann. 2015. A quantitative evaluation of the RAPL power control system. In Proceedings of the 10th International Workshop on Feedback Computing.Google ScholarGoogle Scholar
  45. Huazhe Zhang and Henry Hoffmann. 2016. Maximizing performance under a power cap: A comparison of hardware, software, and hybrid techniques. In Proceedings of the 21st International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS’16). ACM, New York, NY, 545--559. Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. RAPL in Action: Experiences in Using RAPL for Power Measurements

          Recommendations

          Comments

          Login options

          Check if you have access through your login credentials or your institution to get full access on this article.

          Sign in

          Full Access

          • Published in

            cover image ACM Transactions on Modeling and Performance Evaluation of Computing Systems
            ACM Transactions on Modeling and Performance Evaluation of Computing Systems  Volume 3, Issue 2
            Special Issue on ICPE 2017 and Regular Papers
            June 2018
            114 pages
            ISSN:2376-3639
            EISSN:2376-3647
            DOI:10.1145/3199681
            • Editors:
            • Sem Borst,
            • Carey Williamson
            Issue’s Table of Contents

            Copyright © 2018 ACM

            Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

            Publisher

            Association for Computing Machinery

            New York, NY, United States

            Publication History

            • Published: 22 March 2018
            • Accepted: 1 January 2018
            • Revised: 1 October 2017
            • Received: 1 June 2017
            Published in tompecs Volume 3, Issue 2

            Permissions

            Request permissions about this article.

            Request Permissions

            Check for updates

            Qualifiers

            • research-article
            • Research
            • Refereed

          PDF Format

          View or Download as a PDF file.

          PDF

          eReader

          View online with eReader.

          eReader