Haiyun Zhu
ABSTRACT
Due to the popularity of Large Language Models (LLM) in Artificial Intelligence (AI), computational power demands are increasing significantly. Intelligent computing centers that use LLM have growing rapidly. Conventional power consumption measurement methods for servers primarily rely on SPECpower, which utilizes the Java Development Kit (JDK) of standard Java to estimate server performance. By applying different levels of loads to the central processing unit (CPU)-based components, the method estimates power consumption weighting among various performance levels, from which comprehensive performance-to-power consumption ratio is derived. However, these methods cannot access the performance of the graphics processing unit (GPU). Therefore, they are unsuitable for the power consumption test of AI-based intelligent computing servers where GPU component power consumption is predominant. In response, this paper proposes a power consumption measurement architecture and method for LLM-based intelligent computing servers, to evaluate server performance by executing large models and estimating server power consumption by automatic means. The architecture is flexible and thus is able to observe specific components related to AI operations. It provides a feasible solution for power evaluation for intelligent servers that executes LLMs. Through a large-scale testing on servers from various manufacturers, we proved the method is feasible for LLM-based servers and is effective.
- X. Xu, X. Bai and C. Jin, "Research on Power Testing of Servers using Performance-Based Testing Software," Cooling Journal, p. Issue 3, 2021.Google Scholar
- A. Fadwa, "Modeling Power Consumption of Applications Software Running on Servers," University of Waterloo, M.S. Thesis, 2015.Google Scholar
- A. Sawada, H. Kataoka and D. Duolikun, "Power Consumption and Computation Models of a Storage Server," in International Conference on Broadband and Wireless Computing, Communication and Applications, 2015.Google Scholar
- H. Zhu, H. Dai and S. Yang, "Estimating Power Consumption of Servers Using Gaussian Mixture Model," in International Symposium on Computing and Networking, 2017.Google Scholar
- S. Guo, D. Xiang and Z. Zhao, "A Power Measurement Method based on tpc-c Server," Information Technology and Standardization, pp. (9), 3, 2013.Google Scholar
- P. Arabas and M. P. .. Karpowicz, "Server Power Consumption: Measurements and Modeling with MSRs," in International Conference on Automation, 2016.Google Scholar
- R. .. Lent, "A model for network server performance and power consumption," Sustainable Computing Informatics & Systems, pp. 3(2): 80-93. DOI:10.1016/j.suscom.2012.03.004., 2013.Google ScholarCross Ref
- L. D. Gray, A. Kumar and H. H. Li, "Workload Characterization of the SPECpower_ssj2008 Benchmark," in SPEC International Performance Evaluation Workshop, 2008.Google Scholar
- Chen, Jianmin, , "Statistical GPU power analysis using tree-based methods," IEEE Computer Society, pp. 1-6., 2011.Google Scholar
- Ma, X, et. al., "Statistical power consumption analysis and modeling for gpu-based computing," in Workshop on Power Aware Computing and Systems (HotPower ’09), 2009.Google Scholar
- H. F. Wang and Q. K. Chen, "Power Consumption Prediction Model of General-Purpose Computing GPU with Static Program Slicing," Journal of Software, vol. 24, no. 8, pp. 1746-1760, 2014.Google Scholar
- D. Boughzala, L. Lefevre and O. Anne-Cécile, "A macroscopic analysis of GPU power consumption," in COMPAS, 2019.Google Scholar
- Terven, et. al., "Loss Functions and Metrics in Deep Learning," 2023. [Online]. Available: https://arxiv.org/abs/2307.02694.Google Scholar
Index Terms
- A Power Consumption Measurement Method for Large AI-based Intelligent Computing Servers
Recommendations
Performance and Power Consumption Measurement of Java Application Servers
MASCOTS '12: Proceedings of the 2012 IEEE 20th International Symposium on Modeling, Analysis and Simulation of Computer and Telecommunication SystemsIn this paper, we present our in-progress project of modeling performance and power consumption of Java application servers using SPECjEnterprise2010. We run the workload on two application server using two different CPUs, AMD Phenom~II and Intel Atom, ...
A MISO model for power consumption in virtualized servers
Energy efficiency is always a concern in hosting servers. When any new development is added to a host server, the power consumption of the host server must be theoretically and empirically re-evaluated. Because of the ongoing development trends in ...
An Extended Power Consumption Model for Distributed Applications
AINA '12: Proceedings of the 2012 IEEE 26th International Conference on Advanced Information Networking and ApplicationsThe power consumption of information systems, especially servers has to be reduced to realize green eco-society. A client first selects a server in a collection of possible servers and issues a request to the server. The request is performed as a ...
Comments