skip to main content
10.1145/1687399.1687486acmconferencesArticle/Chapter ViewAbstractPublication PagesiccadConference Proceedingsconference-collections
research-article

Adaptive power management using reinforcement learning

Published: 02 November 2009 Publication History

Abstract

System level power management must consider the uncertainty and variability that comes from the environment, the application and the hardware. A robust power management technique must be able to learn the optimal decision from past history and improve itself as the environment changes. This paper presents a novel online power management technique based on model-free constrained reinforcement learning (RL). It learns the best power management policy that gives the minimum power consumption for a given performance constraint without any prior information of workload. Compared with existing machine learning based power management techniques, the RL based learning is capable of exploring the trade-off in the power-performance design space and converging to a better power management policy. Experimental results show that the proposed RL based power management achieves 24% and 3% reduction in power and latency respectively comparing to the existing expert based power management.

References

[1]
L. Benini, A. Bogliolo and G. De Micheli, "A survey of design techniques for system-level dynamic power management," IEEE Trans on VLSI, Vol. 8, Issue 3, pp. 299--316, 2000.
[2]
C. H. Hwang, and A. C. Wu, "A predictive system shutdown method for energy saving of event-driven computation," in Proceedings of International Conference on Computer-Aided Design, Nov. 1997.
[3]
M. Srivastava, A. Chandrakasan, and R. Brodersen, "Predictive system shutdown and other architectural techniques for energy efficient programmable computation," IEEE transactions on Very Large Scale Integration (VLSI) Systems, Vol. 4, pp. 42--55, Mar. 1996.
[4]
L. Benini, G. Paleologo, A. Bogliolo, and G. De Micheli, "Policy optimization for dynamic power management," IEEE transactions on Computer-Aided Design, Vol. 18, pp. 813--833, Jun. 2001.
[5]
T. Simunic, L. Benini, and G. De Micheli, "Event-driven power management," IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems, Vol. 20, pp. 840--857, Jul. 2001.
[6]
H. Jung, and M. Pedram, "Dynamic power management under uncertain information," in Proceedings of Design Automation & Test in Europe (DATE'07), Apr. 2007.
[7]
Q. Qiu, Y. Tan and Q. Wu, "Stochastic Modeling and Optimization for Robust Power Management in a Partially Observable System," in Proceedings of Design Automation & Test in Europe (DATE'07), Apr. 2007.
[8]
Y. Tan, and Q. Qiu, "A framework of stochastic power management using hidden Markov model," in Proceedings of Design Automation & Test in Europe (DATE'08), Apr. 2008.
[9]
G. Theocharous, S. Mannor, N. Shah, P. Gandhi, B. Kveton, S. Siddiqi, and C-H. Yu, "Machine Learning for Adaptive Power Management," Intel Technology Journal, Vol. 10, pp. 299--312, November 2006.
[10]
G. Dhiman, and T. Simunic Rosing, "Dynamic power management using machine learning," in Proceedings of International Conference on Computer-Aided Design (ICCAD'06), Nov. 2006.
[11]
V. L. Prabha and E. C. Monie, "Hardware Architecture of Reinforcement Learning Scheme for Dynamic Power Management in Embedded Systems," EURASIP Journal on Embedded Systems, Vol. 2007.
[12]
Dynamic Power Management in Embedded Systems
[13]
L. Benini, G. Paleologo, A. Bogliolo, and G. De Micheli, "Policy optimization for dynamic power management," IEEE Transactions on Computer-Aided Design, Vol. 18, pp. 813--833, Jun. 1999.
[14]
G. V. Varatkar and R. Marculescu, "On-chip Traffic Modeling and Synthesis for MPEG-2 Video Applications," IEEE Transactions on Very Large Scale Intergration Systems, Vol. 12, No. 1, January 2004.
[15]
M. Pendrith, "On reinforcement learning of control actions in noisy and nonMarkovian domains" Technical Report NSW-CSE-TR-9410, School of Computer Science and Engineering, The University of New South Wales, Sydney, Australia, 1994.
[16]
K. Sikorski and T. Balch, "Model-Based and Model-Free Learning in Markovian and Non-Markovian Environments," in Proceedings of Agents-2001 Workshop on Learning Agents, May 29, 2001.

Cited By

View all
  • (2024)GRES: Guaranteed Remaining Energy Scheduling of Energy-harvesting Sensors by Quality Adaptation2024 13th Mediterranean Conference on Embedded Computing (MECO)10.1109/MECO62516.2024.10577838(1-5)Online publication date: 11-Jun-2024
  • (2023)Smart Knowledge Transfer-based Runtime Power Management2023 Design, Automation & Test in Europe Conference & Exhibition (DATE)10.23919/DATE56975.2023.10136902(1-6)Online publication date: Apr-2023
  • (2023)A Hybrid Linear Programming-Reinforcement Learning Method for Optimal Energy Hub ManagementIEEE Transactions on Smart Grid10.1109/TSG.2022.319745814:1(157-166)Online publication date: Jan-2023
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
ICCAD '09: Proceedings of the 2009 International Conference on Computer-Aided Design
November 2009
803 pages
ISBN:9781605588001
DOI:10.1145/1687399
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 02 November 2009

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. Q-learning
  2. model-free
  3. power management
  4. reinforcement learning

Qualifiers

  • Research-article

Conference

ICCAD '09
Sponsor:

Acceptance Rates

Overall Acceptance Rate 457 of 1,762 submissions, 26%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)30
  • Downloads (Last 6 weeks)3
Reflects downloads up to 15 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2024)GRES: Guaranteed Remaining Energy Scheduling of Energy-harvesting Sensors by Quality Adaptation2024 13th Mediterranean Conference on Embedded Computing (MECO)10.1109/MECO62516.2024.10577838(1-5)Online publication date: 11-Jun-2024
  • (2023)Smart Knowledge Transfer-based Runtime Power Management2023 Design, Automation & Test in Europe Conference & Exhibition (DATE)10.23919/DATE56975.2023.10136902(1-6)Online publication date: Apr-2023
  • (2023)A Hybrid Linear Programming-Reinforcement Learning Method for Optimal Energy Hub ManagementIEEE Transactions on Smart Grid10.1109/TSG.2022.319745814:1(157-166)Online publication date: Jan-2023
  • (2023)Seque: Lean and Energy-aware Data Management for IoT Gateways2023 IEEE International Conference on Edge Computing and Communications (EDGE)10.1109/EDGE60047.2023.00030(133-139)Online publication date: Jul-2023
  • (2023)An Integrated CPU-GPU Frequency Scaling Governor Based on Deep Recurrent Q-Network for Partially Observable Rendering Applications2023 IEEE 5th International Conference on Artificial Intelligence Circuits and Systems (AICAS)10.1109/AICAS57966.2023.10168668(1-5)Online publication date: 11-Jun-2023
  • (2023)Reinforcement learning architecture for cyber–physical–social AI: state-of-the-art and perspectivesArtificial Intelligence Review10.1007/s10462-023-10450-256:11(12655-12688)Online publication date: 22-Mar-2023
  • (2023)Recent Trends in Artificial Intelligence-Inspired Electronic Thermal management—A ReviewRecent Advances in Thermal Sciences and Engineering10.1007/978-981-19-7214-0_13(165-175)Online publication date: 6-May-2023
  • (2022)CARTAD: Compiler-Assisted Reinforcement Learning for Thermal-Aware Task Scheduling and DVFS on MulticoresIEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems10.1109/TCAD.2021.309502841:6(1813-1826)Online publication date: Jun-2022
  • (2022)Deep Reinforcement Learning Enabled Self-Configurable Networks-on-Chip for High-Performance and Energy-Efficient Computing SystemsIEEE Access10.1109/ACCESS.2022.318250010(65339-65354)Online publication date: 2022
  • (2022)Machine learning for a sustainable energy futureNature Reviews Materials10.1038/s41578-022-00490-58:3(202-215)Online publication date: 18-Oct-2022
  • Show More Cited By

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media