Abstract
Deep learning has achieved accuracy and fast training speed and has been successfully applied to many fields, including speech recognition, text processing, image processing and video processing. However, the cost of high power and energy comes together with the high accuracy and training speed of Deep Neural Network (DNN). This inspires researchers to perform characterization in terms of performance, power and energy for guiding the architecture design of DNN models. There are three critical issues to solve for designing a both accurate and energy-efficient DNN model: i) how the software parameters affect the DNN models; ii) how the hardware parameters affect the DNN models; and iii) how to choose the best energy-efficient DNN model. To answer the three issues above, we capture and analyze the performance, power and energy behaviors for multiple experiment settings. We evaluate four DNN models (i.e., LeNet, GoogLeNet, AlexNet, and CaffeNet) with various parameter settings (both hardware and software) on both CPU and GPU platforms. Evaluation results provide detailed DNN characterization and some key insights to facilitate the design of energy-efficient deep learning solutions.
Y. Sun and Z. Ou contributed equally to this work and should be considered as co-first authors.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
cxxnet. https://github.com/dmlc/cxxnet
Imagenet large-scale visual recognition challenge. http://image-net.org/challenges/LSVRC
Nvidia system management interface. https://developer.nvidia.com/nvidia-system-management-interface
Abadi, M., et al.: TensorFlow: large-scale machine learning on heterogeneous distributed systems. ArXiv abs/1603.04467 (2016)
Cai, E., Juan, D.C., Stamoulis, D., Marculescu, D.: NeuralPower: predict and deploy energy-efficient convolutional neural networks. In: The 9th Asian Conference on Machine Learning (ACML 2017) (2017)
Chen, J., et al.: Analyzing time-dimension communication characterizations for representative scientific applications on supercomputer systems. Front. Comput. Sci. 13(6), 1228–1242 (2019)
Chen, T., et al.: MXNet: a flexible and efficient machine learning library for heterogeneous distributed systems. ArXiv abs/1512.01274 (2015)
Collobert, R., Kavukcuoglu, K., Farabet, C.: Torch7: a matlab-like environment for machine learning. In: NIPS 2011 (2011)
Committe, G.: Green500. https://www.top500.org/lists/green500/. Accessed 20 May 2021
Guassic: Text classification with CNN and RNN. https://github.com/gaussic/text-classification-cnn-rnn
Han, S., Pool, J., Tran, J., Dally, W.J.: Learning both weights and connections for efficient neural networks. In: Twenty-ninth Conference on Neural Information Processing Systems (NIPS 2015) (2015)
Hodak, M., Gorkovenko, M., Dholakia, A.: Towards power efficiency in deep learning on data center hardware. In: 2019 IEEE International Conference on Big Data (Big Data) (2019)
Jia, Y., et al.: Caffe: convolutional architecture for fast feature embedding. ArXiv abs/1408.5093 (2014)
Khan, K.N., Hirki, M., Niemi, T., Nurminen, J.K., Ou, Z.: RAPL in action: experiences in using RAPL for power measurements. ACM Trans. Model. Perform. Eval. Comput. Syst. 3(2), 1–26 (2018)
Krizhevsky, A., Sutskever, I., Hinton, G.: ImageNet classification with deep convolutional neural networks. Adv. Neural Inf. Process. Syst. 25(2), 1097–1105 (2012)
LeCun, Y., Boser, B., Denker, J.S., Henderson, D., Howard, R.E.: Backpropagation applied to handwritten zip code recognition. Neural Comput. 1(4), 541–551 (1989)
LeCun, Y., Kavukcuoglu, K., Farabet, C.: Convolutional networks and applications in vision. In: Proceedings of 2010 IEEE International Symposium on Circuits and Systems, pp. 253–256 (2010). https://doi.org/10.1109/ISCAS.2010.5537907
Li, D., Chen, X., Becchi, M., Zong, Z.: Evaluating the energy efficiency of deep convolutional neural networks on CPUs and GPUs. In: 2016 IEEE International Conferences on Big Data and Cloud Computing (BDCloud), Social Computing and Networking (SocialCom), Sustainable Computing and Communications (SustainCom) (BDCloud-SocialCom-SustainCom), pp. 477–484 (2016)
Sun, M., Li, J., Guo, Z.: THUCTC: an efficient Chinese text classification toolkit. http://thuctc.thunlp.org/
Ou, Z., Chen, J., Zhang, Y., Dong, Y., Yuan, Y., Wang, Z.: Power modeling for Phytium FT-2000+/64 multi-core architecture. In PPoPP 2020 Workshop: Principles and Practice of Parallel Programming 2020, Workshop: Benchmarking in the Datacenter, 7 p. (2020)
Rodrigues, C.F., Riley, G., Luján, M.: Fine-grained energy profiling for deep convolutional neural networks on the Jetson TX1. In: 2017 IEEE International Symposium on Workload Characterization (IISWC), pp. 114–115 (2017)
Rouhani, B.D., Mirhoseini, A., Koushanfar, F.: DeLight: adding energy dimension to deep neural networks. In: International Symposium on Low Power Electronics & Design (2016)
Sutskever, I., Vinyals, O., Le, Q.V.: Sequence to sequence learning with neural networks. In: Proceedings of the 27th International Conference on Neural Information Processing Systems, NIPS 2014, vol. 2, p. 3104–3112. MIT Press, Cambridge (2014)
Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Rabinovich, A.: Going deeper with convolutions. IEEE Computer Society (2014)
Tang, Z., Wang, Y., Wang, Q., Chu, X.: The impact of GPU DVFs on the energy and performance of deep learning: an empirical study. In: The Tenth ACM International Conference (2019)
Thomas, D., Shanmugasundaram, M.: A survey on different overclocking methods. In: 2018 Second International Conference on Electronics, Communication and Aerospace Technology (ICECA), pp. 1588–1592 (2018)
Wu, F., et al.: A holistic energy-efficient approach for a processor-memory system. Tsinghua Sci. Technol. 24, 468–483 (2019)
Yang, T.J., Chen, Y.H., Sze, V.: Designing energy-efficient convolutional neural networks using energy-aware pruning. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2017)
Yao, C., et al.: Evaluating and analyzing the energy efficiency of CNN inference on high-performance GPU. Pract. Exp. Concurr. Comput. 33, e6064 (2020)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Sun, Y. et al. (2021). Evaluating Performance, Power and Energy of Deep Neural Networks on CPUs and GPUs. In: Cai, Z., Li, J., Zhang, J. (eds) Theoretical Computer Science. NCTCS 2021. Communications in Computer and Information Science, vol 1494. Springer, Singapore. https://doi.org/10.1007/978-981-16-7443-3_12
Download citation
DOI: https://doi.org/10.1007/978-981-16-7443-3_12
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-16-7442-6
Online ISBN: 978-981-16-7443-3
eBook Packages: Computer ScienceComputer Science (R0)
