Abstract
To improve energy efficiency and comply with the power budgets, it is important to be able to measure the power consumption of cloud computing servers. Intel’s Running Average Power Limit (RAPL) interface is a powerful tool for this purpose. RAPL provides power limiting features and accurate energy readings for CPUs and DRAM, which are easily accessible through different interfaces on large distributed computing systems. Since its introduction, RAPL has been used extensively in power measurement and modeling. However, the advantages and disadvantages of RAPL have not been well investigated yet. To fill this gap, we conduct a series of experiments to disclose the underlying strengths and weaknesses of the RAPL interface by using both customized microbenchmarks and three well-known application level benchmarks: Stream, Stress-ng, and ParFullCMS. Moreover, to make the analysis as realistic as possible, we leverage two production-level power measurement datasets from the Taito, a supercomputing cluster of the Finnish Center of Scientific Computing and also replicate our experiments on Amazon EC2. Our results illustrate different aspects of RAPL and document the findings through comprehensive analysis. Our observations reveal that RAPL readings are highly correlated with plug power, promisingly accurate enough, and have negligible performance overhead. Experimental results suggest RAPL can be a very useful tool to measure and monitor the energy consumption of servers without deploying any complex power meters. We also show that there are still some open issues, such as driver support, non-atomicity of register updates, and unpredictable timings that might weaken the usability of RAPL in certain scenarios. For such scenarios, we pinpoint solutions and workarounds.
- VMSTAT. Retrieved from http://www.linuxcommand.org/man_pages/vmstat8.html.Google Scholar
- David Abdurachmanov, Peter Elmer, Giulio Eulisse, Robert Knight, Tapio Niemi, Jukka K. Nurminen, Filip Nyback, Goncalo Pestana, Zhonghong Ou, and Kashif Nizam Khan. 2014. Techniques and tools for measuring energy efficiency of scientific software applications. CoRR abs/1410.3440 (2014). Retrieved from http://arxiv.org/abs/1410.3440.Google Scholar
- Xi Chen, Chi Xu, Robert P. Dick, and Zhuoqing Morley Mao. 2010. Performance and power modeling in a multi-programmed multi-core environment. In Proceedings of the 47th Design Automation Conference (DAC’10). ACM, New York, NY, 813--818. Google ScholarDigital Library
- CSC. 2017. Taito supercluster. Retrieved from https://research.csc.fi/taito-supercluster.Google Scholar
- Howard David, Eugene Gorbatov, Ulf R. Hanebutte, Rahul Khanna, and Christian Le. 2010. RAPL: Memory power estimation and capping. In Proceedings of the 16th ACM/IEEE International Symposium on Low Power Electronics and Design (ISLPED’10). ACM, New York, NY, 189--194. Google ScholarDigital Library
- Spencer Desrochers, Chad Paradis, and Vincent M. Weaver. 2016. A validation of DRAM RAPL power measurements. In Proceedings of the 2nd International Symposium on Memory Systems (MEMSYS’16). ACM, New York, NY, 455--470. Google ScholarDigital Library
- Mohammed El Mehdi Diouri, Manuel F. Dolz, Olivier Glück, Laurent Lefèvre, Pedro Alonso, Sandra Catalán, Rafael Mayo, and Enrique S. Quintana-Ortí. 2014. Assessing power monitoring approaches for energy and power analysis of computers. Sustain. Comput.: Informat. Syst. 4, 2 (2014), 68--82.Google ScholarCross Ref
- Jack Dongarra, Hatem Ltaief, Piotr Luszczek, and Vincent M. Weaver. 2012. Energy footprint of advanced dense numerical linear algebra using tile algorithms on multicore architectures. In Proceedings of the 2012 2nd International Conference on Cloud and Green Computing (CGC’12). IEEE, 274--281. Google ScholarDigital Library
- Amazon EC2. Instance Types. Retrieved from https://aws.amazon.com/ec2/instance-types/.Google Scholar
- S. Agostinelli et al. 2003. Geant4- A simulation toolkit. Nucl. Instrum. Methods Phys. Res. Sec. A: Accel. Spectrom. Detect. Assoc. Equip. 506, 3 (2003), 250--303.Google ScholarCross Ref
- Daniel Hackenberg, Thomas Ilsche, Robert Schone, Daniel Molka, Maik Schmidt, and Wolfgang E Nagel. 2013. Power measurement techniques on standard compute nodes: A quantitative comparison. In Proceedings of the 2013 IEEE International Symposium on Performance Analysis of Systems and Software (ISPASS’13). IEEE, 194--204.Google ScholarCross Ref
- Daniel Hackenberg, Robert Schöne, Thomas Ilsche, Daniel Molka, Joseph Schuchart, and Robin Geyer. 2015. An energy efficiency feature survey of the intel haswell processor. In Proceedings of the International Parallel and Distributed Processing Symposium (IPDPS’15). Google ScholarDigital Library
- D. Hackenberg, R. Schöne, T. Ilsche, D. Molka, J. Schuchart, and R. Geyer. 2015. An energy efficiency feature survey of the intel haswell processor. In Proceedings of the 2015 IEEE International Parallel and Distributed Processing Symposium Workshop. 896--904. Google ScholarDigital Library
- Marcus Hähnel, Björn Döbel, Marcus Völp, and Hermann Härtig. 2012. Measuring energy consumption for short code paths using RAPL. ACM SIGMETRICS Perform. Eval. Rev. 40, 3 (2012), 13--17. Google ScholarDigital Library
- Mikael Hirki. 2015. Energy and Performance Profiling of Scientific Computing. Master’s thesis. Aalto University.Google Scholar
- Mikael Hirki. 2017. RAPL Testing and Instruction Decoder Benchmarks. Retrieved from https://github.com/mhirki/idq-bench2.Google Scholar
- Mikael Hirki, Zhonghong Ou, Kashif Nizam Khan, Jukka K Nurminen, and Tapio Niemi. 2016. Empirical study of the power consumption of the x86-64 instruction decoder. In Proceedings of the USENIX Workshop on Cool Topics on Sustainable Data Centers (CoolDC’16). USENIX Association.Google Scholar
- Song Huang, Michael Lang, Scott Pakin, and Song Fu. 2015. Measurement and characterization of haswell power and energy consumption. In Proceedings of the 3rd International Workshop on Energy Efficient Supercomputing (E2SC’15). ACM, New York, NY. Google ScholarDigital Library
- T. Ilsche, D. Hackenberg, S. Graul, R. Schöne, and J. Schuchart. 2015. Power measurements for compute nodes: Improving sampling rates, granularity and accuracy. In Proceedings of the 2015 6th International Green and Sustainable Computing Conference (IGSC’15). 1--8. Google ScholarDigital Library
- Intel Corporation 2015. Intel® 64 and IA-32 Architectures Software Developer’s Manual, Volume 3, System Programming Guide. Intel Corporation.Google Scholar
- R. Kavanagh, D. Armstrong, and K. Djemame. 2016. Accuracy of energy model calibration with IPMI. In Proceedings of the 2016 IEEE 9th International Conference on Cloud Computing (CLOUD’16). 648--655.Google Scholar
- Jaimie Kelley, Christopher Stewart, Devesh Tiwari, and Saurabh Gupta. 2016. Adaptive power profiling for many-core HPC architectures. In Proceedings of the International Conference on Autonomic Computing.Google ScholarCross Ref
- K. N. Khan, F. Nybäck, Z. Ou, J. K. Nurminen, T. Niemi, G. Eulisse, P. Elmer, and D. Abdurachmanov. 2015. Energy profiling using IgProf. In Proceedings of the 2015 15th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing. 1115--1118.Google Scholar
- Kashif Nizam Khan, Zhonghong Ou, Mikael Hirki, Jukka K. Nurminen, and Tapio Niemi. 2016. How much power does your server consume? Estimating wall socket power using RAPL measurements. Comput. Sci. Res. Dev. 31, 4 (2016), 207--214. Google ScholarDigital Library
- R. Khanna, F. Zuhayri, M. Nachimuthu, C. Le, and M. J. Kumar. 2011. Unified extensible firmware interface: An innovative approach to DRAM power control. In Proceedings of the 2011 International Conference on Energy Aware Computing. 1--6.Google Scholar
- Gary Lawson, Masha Sosonkina, and Yuzhong Shen. 2015. Towards modeling energy consumption of Xeon Phi. arXiv:1505.06539 (2015).Google Scholar
- H. Liu. 2011. A measurement study of server utilization in public clouds. In Proceedings of the 2011 IEEE 9th International Conference on Dependable, Autonomic and Secure Computing. 435--442. Google ScholarDigital Library
- Ioannis Manousakis, Foivos S. Zakkak, Polyvios Pratikakis, and Dimitrios S. Nikolopoulos. 2015. TProf: An energy profiler for task-parallel programs. Sustainable Computing: Informatics and Systems 5 (2015), 1--13. http://www.sciencedirect.com/science/article/pii/S2210537914000390.Google ScholarCross Ref
- John D. McCalpin. 1995. Memory bandwidth and machine balance in current high performance computers. IEEE Comput. Soc. Tech. Committee Comput. Architect. (TCCA) Newsletter (Dec. 1995), 19--25.Google Scholar
- John C. McCullough, Yuvraj Agarwal, Jaideep Chandrashekar, Sathyanarayan Kuppuswamy, Alex C. Snoeren, and Rajesh K. Gupta. 2011. Evaluating the effectiveness of model-based power characterization. In Proceedings of the 2011 USENIX Conference on USENIX Annual Technical Conference (USENIXATC’11). USENIX Association, Berkeley, CA, 12--12. Retrieved from DOI: http://dl.acm.org/citation.cfm?id=2002181.2002193. Google ScholarDigital Library
- C. Möbius, W. Dargie, and A. Schill. 2014. Power consumption estimation models for processors, virtual machines, and servers. IEEE Trans. Parallel Distrib. Syst. 25, 6 (June 2014), 1600--1614. Google ScholarDigital Library
- Chad M. Paradis. 2015. Detailed Low-cost Energy and Power Monitoring of Computing Systems. Master’s thesis. University of Maine.Google Scholar
- Tapasya Patki, David K. Lowenthal, Barry Rountree, Martin Schulz, and Bronis R. de Supinski. 2013. Exploring hardware overprovisioning in power-constrained, high performance computing. In Proceedings of the 27th International ACM Conference on International Conference on Supercomputing (ICS’13). ACM, New York, NY, 173--182. Google ScholarDigital Library
- Plugwise. 2017. Energy management systems of the 21st century. Retrieved from https://www.plugwise.com/.Google Scholar
- Harald Servat, Germán Llort, Judit Giménez, and Jesús Labarta. 2016. Detailed and simultaneous power and performance analysis. Concurrency and Computation: Practice and Experience 28, 2 (2016), 252--273. Google ScholarDigital Library
- A. Skrenes and C. Williamson. 2016. Experimental calibration and validation of a speed scaling simulator. In Proceedings of the 2016 IEEE 24th International Symposium on Modeling, Analysis and Simulation of Computer and Telecommunication Systems (MASCOTS’16). 105--114.Google Scholar
- Balaji Subramaniam and Wu-chun Feng. 2015. On the energy proportionality of scale-out workloads. CoRR abs/1501.02729 (2015). Retrieved from DOI: http://arxiv.org/abs/1501.02729.Google Scholar
- Balaji Subramaniam and Wu-chun Feng. 2013. Towards energy-proportional computing for enterprise-class server workloads. In Proceedings of the 4th ACM/SPEC International Conference on Performance Engineering (ICPE’13). ACM, New York, NY, 15--26. Google ScholarDigital Library
- Vincent Weaver. 2015. rapl-read.c. Retrieved from http://web.eece.maine.edu/ vweaver/projects/rapl/rapl-read.c.Google Scholar
- V. M. Weaver, M. Johnson, K. Kasichayanula, J. Ralph, P. Luszczek, D. Terpstra, and S. Moore. 2012. Measuring energy and power with PAPI. In Proceedings of the 2012 41st International Conference on Parallel Processing Workshops. 262--268. Google ScholarDigital Library
- Vincent M. Weaver, Matt Johnson, Kiran Kasichayanula, James Ralph, Piotr Luszczek, Dan Terpstra, and Shirley Moore. 2012. Measuring energy and power with PAPI. In Proceedings of the 2012 41st International Conference on Parallel Processing Workshops (ICPPW’12). IEEE Computer Society, Washington, DC, 262--268. Google ScholarDigital Library
- Simon N. Wood. 2017. Generalized Additive Models: An Introduction with R. CRC Press.Google ScholarDigital Library
- Yan Zhai, Xiao Zhang, Stephane Eranian, Lingjia Tang, and Jason Mars. 2014. HaPPy: Hyperthread-aware power profiling dynamically. In Proceedings of the 2014 USENIX Annual Technical Conference (USENIXATC’14). USENIX Association, Philadelphia, PA, 211--217. Google ScholarDigital Library
- Huazhe Zhang and Henry Hoffmann. 2015. A quantitative evaluation of the RAPL power control system. In Proceedings of the 10th International Workshop on Feedback Computing.Google Scholar
- Huazhe Zhang and Henry Hoffmann. 2016. Maximizing performance under a power cap: A comparison of hardware, software, and hybrid techniques. In Proceedings of the 21st International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS’16). ACM, New York, NY, 545--559. Google ScholarDigital Library
Index Terms
- RAPL in Action: Experiences in Using RAPL for Power Measurements
Recommendations
A Validation of DRAM RAPL Power Measurements
MEMSYS '16: Proceedings of the Second International Symposium on Memory SystemsRecent Intel processors support the Running Average Power Level (RAPL) interface, which among other things provides estimated energy measurements for the CPUs, integrated GPU, and DRAM. These measurements are easily accessible by the user, and can be ...
Measuring energy consumption for short code paths using RAPL
Measuring the energy consumption of software components is a major building block for generating models that allow for energy-aware scheduling, accounting and budgeting. Current measurement techniques focus on coarse-grained measurements of application ...
RAPL: memory power estimation and capping
ISLPED '10: Proceedings of the 16th ACM/IEEE international symposium on Low power electronics and designThe drive for higher performance and energy efficiency in data-centers has influenced trends toward increased power and cooling requirements in the facilities. Since enterprise servers rarely operate at their peak capacity, efficient power capping is ...
Comments