Yuan Cao
ABSTRACT
At the era of Artificial Intelligence and Internet of Things (AIoT), battery-powered mobile devices are required to perform more sophisticated tasks featured with fast varying workloads and constrained power supply, demanding more efficient run-time power management. In this paper, we propose a deep reinforcement learning framework for dynamic power and thermal co-management. We build several machine learning models that incorporate the physical details for an ARM Cortex-A72, with on average 3% and 1% error for power and temperature predictions, respectively. We then build an efficient deep reinforcement learning control incorporating the machine learning models and facilitating the run-time dynamic voltage and frequency scaling (DVFS) strategy selection based on the predicted power, workloads and temperature. We evaluate our proposed framework, and compare the performance with existing management methods. The results suggest that our proposed framework can achieve 6.8% performance improvement compared with other alternatives.
Supplemental Material
- H. Huang et al. Autonomous power management with double-q reinforcement learning method. IEEE TII, 16(3):1938--1946, 2020.Google Scholar
- Shervin Hajiamini et al. A dynamic programming framework for dvfs-based energy-efficiency in multicore systems. IEEE TSC, 5(1):1--12, 2020.Google Scholar
- Camé lia Slimani et al. Hymad: a hybrid memory-aware DVFS strategy. ACM SIGBED Review, 16(3):45--50, 2019. Google ScholarDigital Library
- C. Zhuo et al. Noise-aware dvfs for efficient transitions on battery-powered iot devices. IEEE TCAD, 39(7):1498--1510, 2020.Google Scholar
- U. R. Tida et al. Dynamic frequency scaling aware opportunistic through-silicon-via inductor utilization in resonant clocking. IEEE TCAD, 39(2):281--293, 2020.Google Scholar
- J. Charles et al. Evaluation of the intel® core? i7 turbo boost feature. In Proc. IISWC, pages 188--197, 2009. Google ScholarDigital Library
- Dimitrios Chasapis et al. Power efficient job scheduling by predicting the impact of processor manufacturing variability. In Proc. ICS, pages 296--307, 2019. Google ScholarDigital Library
- J. Deng et al. Energy-efficient real-time uav object detection on embedded platforms. IEEE TCAD, 39(10):3123--3127, 2020.Google Scholar
- Zhongyang Liu et al. A multi-level-optimization framework for fpga-based cellular neural network implementation. ACM JETCS, 14(4):1--17, 2018. Google ScholarDigital Library
- Chi-Hsien Pao et al. XGBIR: an xgboost-based IR drop predictor for power delivery network. In Proc. DATE, pages 1307--1310, 2020. Google ScholarDigital Library
- Ganapati Bhat et al. Algorithmic optimization of thermal and power management for heterogeneous mobile platforms. IEEE TVLSI, 26(3):544--557, 2018. Google ScholarDigital Library
- Somdip Dey et al. P-edgecoolingmode: an agent-based performance aware thermal management unit for DVFS enabled heterogeneous mpsocs. IET CDT, 13(6):514--523, 2019.Google Scholar
- C. Zhuo et al. From layout to system: Early stage power delivery and architecture co-exploration. IEEE TCAD, 39(7):1291--1304, 2019.Google Scholar
- Canturk Isci et al. An analysis of efficient multi-core global power management policies: Maximizing performance for a given power budget. In Proc. MICRO, pages 347--358, 2006. Google ScholarDigital Library
- P. Meng et al. Multi-optimization power management for chip multiprocessors. In Proc. PACT, pages 177--186, 2008. Google ScholarDigital Library
- Matthew Walker et al. Accurate and stable empirical cpu power modelling for multi-and many-core systems. 2018.Google Scholar
- Efraim Rotem et al. Power and thermal constraints of modern system-on-a-chip computer. In Proc. THERMINIC, pages 141--146, 2013.Google Scholar
- Longyang Lin et al. Integrated power management for battery-indifferent systems with ultra-wide adaptation down to nw. IEEE JSSC, 55(4):967--976, 2020.Google Scholar
- R. Teodorescu et al. Variation-aware application scheduling and power management for chip multiprocessors. ACM SIGARCH CAN, 36(3):363--374, 2008. Google ScholarDigital Library
- Volodymyr Mnih et al. Human-level control through deep reinforcement learning. Nature, 518(7540):529--533, 2015.Google ScholarCross Ref
- Nathan Binkert et al. The gem5 simulator. ACM SIGARCH CAN, 39(2):1--7, 2011. Google ScholarDigital Library
- Sheng Li et al. Mcpat: an integrated power, area, and timing modeling framework for multicore and manycore architectures. In Proc. MICRO, pages 469--480., 2009. Google ScholarDigital Library
- Wei Huang et al. Hotspot: A compact thermal modeling methodology for early-stage VLSI design. IEEE TVLSI, 14(5):501--513, 2006. Google ScholarDigital Library
- Jó akim von Kistowski et al. Analysis of the influences on server power consumption and energy efficiency for cpu-intensive workloads. In Proc. ICPE, pages 223--234, 2015. Google ScholarDigital Library
- Raid Ayoub and Tajana Rosing. Cool and save: cooling aware dynamic workload scheduling in multi-socket CPU systems. In Proc. ASP-DAC, pages 891--896., 2010. Google ScholarDigital Library
- John Henning. SPEC CPU2006 benchmark descriptions. ACM SIGARCH CAN, 34(4):1--17, 2006. Google ScholarDigital Library
- Kevin Skadron et al. Temperature-aware microarchitecture: Modeling and implementation. IEEE TACO, 1(1):94--125, 2004. Google ScholarDigital Library
- Shervin Sharifi and Tajana Simunic Rosing. Accurate direct and indirect on-chip temperature sensing for efficient dynamic thermal management. IEEE TCAD, 29(10):1586--1599, 2010. Google ScholarDigital Library
- J. Millard and Ludwik Kurz. The kolmogorov-smirnov tests in signal detection (corresp.). IEEE TIT, 13(2):341--342, 1967. Google ScholarDigital Library
- Ben Krö se. Learning from delayed rewards. Robotics Auton. Syst., 15(4):233--235, 1995.Google ScholarCross Ref
- Hado van Hasselt et al. Deep reinforcement learning with double q-learning. CoRR, abs/1509.06461, 2015.Google Scholar
- Xusheng Zhan et al. PARSEC3.0: A multicore benchmark suite with network stacks and SPLASH-2X. ACM SIGARCH CAN, 44(5):1--16, 2016. Google ScholarDigital Library
- Ana Madureira et al. Negotiation mechanism for self-organized scheduling system with collective intelligence. Neurocomputing, 132:97--110, 2014. Google ScholarDigital Library
- Venkatesh Pallipadi and Alexey Starikovskiy. The ondemand governor. In Proceedings of the Linux Symposium, 2(00216):215--230, 2006.Google Scholar
Index Terms
- An Efficient and Flexible Learning Framework for Dynamic Power and Thermal Co-Management
Recommendations
Application and Thermal-reliability-aware Reinforcement Learning Based Multi-core Power Management
Special Issue on HALO for Energy-Constrained On-Chip Machine Learning, Part 2 and Regular PapersPower management through dynamic voltage and frequency scaling (DVFS) is one of the most widely adopted techniques. However, it impacts application reliability (due to soft errors, circuit aging, and deadline misses). However, increased power density ...
Managing power constraints in a single-core scenario through power tokens
Current microprocessors face constant thermal and power-related problems during their everyday use, usually solved by applying a power budget to the processor/core. Dynamic voltage and frequency scaling (DVFS) has been an effective technique that ...
Dynamic Power Management of Voltage-Frequency Island Partitioned Networks-on-Chip using Intel Sing-Chip Cloud Computer
NOCS '11: Proceedings of the Fifth ACM/IEEE International Symposium on Networks-on-ChipContinuous technology scaling has enabled the integration of multiple cores on the same chip. To overcome the disadvantages of buses, the Network-on-Chip (NoC) architecture has been proposed as a new communication paradigm. To further mitigate the ...
Comments