Abstract
With the rise of artificial intelligence (AI) in recent years and the subsequent increase in complexity of the applied models, the growing demand in computational resources is starting to pose a significant challenge. The need for higher compute power is being met with increasingly more potent accelerator hardware as well as the use of large and powerful compute clusters. However, the gain in prediction accuracy from large models trained on distributed and accelerated systems ultimately comes at the price of a substantial increase in energy demand, and researchers have started questioning the environmental friendliness of such AI methods at scale. Consequently, awareness of energy efficiency plays an important role for AI model developers and hardware infrastructure operators likewise. The energy consumption of AI workloads depends both on the model implementation and the composition of the utilized hardware. Therefore, accurate measurements of the power draw of respective AI workflows on different types of compute nodes is key to algorithmic improvements and the design of future compute clusters and hardware. Towards this end, we present measurements of the energy consumption of two typical applications of deep learning models on different types of heterogeneous compute nodes. Our results indicate that 1. contrary to common approaches, deriving energy consumption directly from runtime is not accurate, but the consumption of the compute node needs to be considered regarding its composition; 2. neglecting accelerator hardware on mixed nodes results in overproportional inefficiency regarding energy consumption; 3. energy consumption of model training and inference should be considered separately – while training on GPUs outperforms all other node types regarding both runtime and energy consumption, inference on CPU nodes can be comparably efficient. One advantage of our approach is the fact that the information on energy consumption is available to all users of the supercomputer and not just those with administrator rights, enabling an easy transfer to other workloads alongside a raise in user-awareness of energy consumption.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Brown, T., et al.: Language models are few-shot learners. Adv. Neural Inf. Process. Syst. 33, 1877–1901 (2020)
David, H., Gorbatov, E., Hanebutte, U.R., Khanna, R., Le, C.: RAPL: memory power estimation and capping. In: 2010 ACM/IEEE International Symposium on Low-Power Electronics and Design (ISLPED), pp. 189–194. IEEE (2010)
Devlin, J., Chang, M.W., Lee, K., Toutanova, K.: BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding (2018). https://doi.org/10.48550/ARXIV.1810.04805, URL https://arxiv.org/abs/1810.04805
Endrei, M., et al.: Energy efficiency modeling of parallel applications. In: SC18: International Conference for High Performance Computing, Networking, Storage and Analysis, pp. 212–224 (2018). https://doi.org/10.1109/SC.2018.00020
Ezzatti, P., Quintana-Ortí, E.S., Remón, A., Saak, J.: Power-aware computing (2019)
Gholkar, N., Mueller, F., Rountree, B., Marathe, A.: Pshifter: feedback-based dynamic power shifting within HPC jobs for performance. In: Proceedings of the 27th International Symposium on High-Performance Parallel and Distributed Computing, pp. 106–117, HPDC 2018, Association for Computing Machinery, New York, NY, USA (2018). ISBN 9781450357852, https://doi.org/10.1145/3208040.3208047
Hodak, M., Gorkovenko, M., Dholakia, A.: Towards power efficiency in deep learning on data center hardware. In: 2019 IEEE International Conference on Big Data (Big Data), pp. 1814–1820. IEEE (2019)
Ilsche, T., et al.: Power measurement techniques for energy-efficient computing: reconciling scalability, resolution, and accuracy. SICS Softw.-Intens. Cyber-Phys. Syst. 34(1), 45–52 (2018). https://doi.org/10.1007/s00450-018-0392-9
Jumper, J., et al.: Highly accurate protein structure prediction with AlphaFold. Nature 596(7873), 583–589 (2021)
Kasichayanula, K., Terpstra, D., Luszczek, P., Tomov, S., Moore, S., Peterson, G.D.: Power aware computing on GPUs. In: 2012 Symposium on Application Accelerators in High Performance Computing, pp. 64–73. IEEE (2012)
Kong, W., Dong, Z.Y., Jia, Y., Hill, D.J., Xu, Y., Zhang, Y.: Short-term residential load forecasting based on LSTM recurrent neural network. IEEE Trans. Smart Grid 10(1), 841–851 (2019). ISSN 1949–3061, https://doi.org/10.1109/TSG.2017.2753802
Lacoste, A., Luccioni, A., Schmidt, V., Dandres, T.: Quantifying the carbon emissions of machine learning. arXiv preprint arXiv:1910.09700 (2019)
Li, D., Chen, X., Becchi, M., Zong, Z.: Evaluating the energy efficiency of deep convolutional neural networks on CPUs and GPUs. In: 2016 BDCloud-SocialCom-SustainCom, pp. 477–484. IEEE (2016)
Montgomerie-Corcoran, A., Venieris, S.I., Bouganis, C.S.: Power-aware FPGA mapping of convolutional neural networks. In: 2019 International Conference on Field-Programmable Technology (ICFPT), pp. 327–330. IEEE (2019)
Muzaffar, S., Afshari, A.: Short-term load forecasts using LSTM networks. Energy Procedia 158, 2922–2927 (2019). ISSN 1876–6102, https://doi.org/10.1016/j.egypro.2019.01.952, https://www.sciencedirect.com/science/article/pii/S1876610219310008
Nonaka, J., Hanawa, T., Shoji, F.: Analysis of cooling water temperature impact on computing performance and energy consumption. In: 2020 IEEE International Conference on Cluster Computing (CLUSTER), pp. 169–175 (2020). https://doi.org/10.1109/CLUSTER49012.2020.00027
NVIDIA Corporation: NVML API Reference. https://docs.nvidia.com/deploy/nvml-api/index.html (2022). Accessed 03 Apr 2022
Pakin, S., et al.: Power usage of production supercomputers and production workloads. Concurr. Comput. Pract. Exper. 28(2), 274–290 (2016). https://doi.org/10.1002/cpe.3191,https://onlinelibrary.wiley.com/doi/abs/10.1002/cpe.3191
Paszke, A., Gross, S., Massa, F., Lerer, A., et al.: Pytorch: an imperative style, high-performance deep learning library. Adv. Neural Inf. Process. Syst. 32, 8024–8035, Curran Associates, Inc. (2019). http://papers.neurips.cc/paper/9015-pytorch-an-imperative-style-high-performance-deep-learning-library.pdf
Patel, T., Liu, Z., Kettimuthu, R., Rich, P.M., Allcock, W.E., Tiwari, D.: Job characteristics on large-scale systems: long-term analysis, quantification, and implications. In: SC20: International Conference for High Performance Computing, Networking, Storage and Analysis, pp. 1–17 (2020)
Patel, T., Wagenhäuser, A., Eibel, C., Hönig, T., Zeiser, T., Tiwari, D.: What does power consumption behavior of HPC jobs reveal?: demystifying, quantifying, and predicting power consumption characteristics. In: 2020 IEEE International Parallel and Distributed Processing Symposium (IPDPS), pp. 799–809 (2020). https://doi.org/10.1109/IPDPS47924.2020.00087
Raucent, F.: Western Europe Power Consumption Dataset. https://www.kaggle.com/datasets/francoisraucent/western-europe-power-consumption (2020)
Schwartz, R., Dodge, J., Smith, N.A., Etzioni, O.: Green AI. Commun. ACM 63(12), 54–63 (2020)
Shin, W., Oles, V., Karimi, A.M., Ellis, J.A., Wang, F.: Revealing power, energy and thermal dynamics of a 200pf pre-exascale supercomputer. In: Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis, SC 2021, Association for Computing Machinery, New York, NY, USA (2021). ISBN 9781450384421, https://doi.org/10.1145/3458817.3476188, https://doi.org/10.1145/3458817.3476188
Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 (2014)
Strubell, E., Ganesh, A., McCallum, A.: Energy and policy considerations for deep learning in NLP. arXiv preprint arXiv:1906.02243 (2019)
Suda, R., et al.: Accurate measurements and precise modeling of power dissipation of CUDA kernels toward power optimized high performance CPU-GPU computing. In: 2009 International Conference on Parallel and Distributed Computing, Applications and Technologies, pp. 432–438. IEEE (2009)
TOP500.org: Green500 List - June 2022. https://www.top500.org/lists/green500/2022/06/ (2022). Accessed 01 July 2022
WG, E.H.: Energy Efficient High Performance Computing Working Group. https://eehpcwg.llnl.gov/ (2022). Accessed 01 July 2022
Wong, A.: COVID-Net Open Initiative. https://alexswong.github.io/COVID-Net/ (2020). Accessed 29 Mar 2022
Zhang, H., Hoffmann, H.: Podd: power-capping dependent distributed applications. In: Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis, SC 2019, Association for Computing Machinery, New York, NY, USA (2019). ISBN 9781450362290, https://doi.org/10.1145/3295500.3356174
Zhao, A.: COVIDx CXR-2 (2021). https://www.kaggle.com/datasets/andyczhao/covidx-cxr2/
Acknowledgment
This work was performed on the HoreKa supercomputer funded by the Ministry of Science, Research and the Arts Baden-Württemberg and by the Federal Ministry of Education and Research.
This work is supported by the Helmholtz Association Initiative and Networking Fund under the Helmholtz AI, HIDSS4Health, Helmholtz Imaging and the Helmholtz Metadata Collaboration platform grants and the HAICORE@KIT partition.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 Springer Nature Switzerland AG
About this paper
Cite this paper
Caspart, R. et al. (2022). Precise Energy Consumption Measurements of Heterogeneous Artificial Intelligence Workloads. In: Anzt, H., Bienz, A., Luszczek, P., Baboulin, M. (eds) High Performance Computing. ISC High Performance 2022 International Workshops. ISC High Performance 2022. Lecture Notes in Computer Science, vol 13387. Springer, Cham. https://doi.org/10.1007/978-3-031-23220-6_8
Download citation
DOI: https://doi.org/10.1007/978-3-031-23220-6_8
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-23219-0
Online ISBN: 978-3-031-23220-6
eBook Packages: Computer ScienceComputer Science (R0)