skip to main content
10.1145/3607947.3608004acmotherconferencesArticle/Chapter ViewAbstractPublication Pagesic3Conference Proceedingsconference-collections
research-article

A Novel Model based Energy Management Strategy for Plug-in Hybrid Electric Vehicles using Deep Reinforcement Learning

Published: 28 September 2023 Publication History

Abstract

Over the last few years, Hybrid Electric Vehicles (HEVs) have become increasingly popular due to their potential to simplify fuel consumption and greenhouse gas emissions. The energy management of HEVs is a critical task that involves controlling the power split between the Internal Combustion Engine (ICE) and electric motor based on the vehicle’s state and driving conditions. Traditional rule-based strategies for HEV energy management may not be able to adapt to varying driving conditions or optimize the vehicle’s performance in real-time. To address this, researchers have explored the potential of advanced machine learning techniques, such as Deep Reinforcement Learning (DRL), as a more effective approach for HEV energy management. DRL is a subfield of machine learning that combines deep neural networks with reinforcement learning to learn an optimal control policy. Among various DRL algorithms, Deep Dyna-Q learning is a hybrid approach that combines model-based and model-free learning. Our paper introduces an innovative strategy for energy management for HEVs using Deep Dyna-Q learning that optimizes the power split in real-time based on the vehicle’s state and driving conditions. We evaluate the proposed strategy on two driving cycles and compare it with Deep Q-Learning (DQL). The findings indicate that the energy management approach presented in this paper surpasses DQL in terms of vehicle performance and fuel efficiency for both driving cycles.

References

[1]
Guodong Du, Yuan Zou, Xudong Zhang, Teng Liu, Jinlong Wu, and Dingbo He. 2020. Deep reinforcement learning based energy management for a hybrid electric vehicle. Energy 201 (2020), 117591.
[2]
Guodong Du, Yuan Zou, Xudong Zhang, Teng Liu, Jinlong Wu, and Dingbo He. 2020. Deep reinforcement learning based energy management for a hybrid electric vehicle. Energy 201 (2020), 117591.
[3]
Xiaosong Hu, Teng Liu, Xuewei Qi, and Matthew Barth. 2019. Reinforcement learning for hybrid and plug-in hybrid electric vehicle energy management: Recent advances and prospects. IEEE Industrial Electronics Magazine 13, 3 (2019), 16–25.
[4]
M Kandidayeni, JP Trovão, M Soleymani, and L Boulon. 2022. Towards health-aware energy management strategies in fuel cell hybrid electric vehicles: A review. International Journal of Hydrogen Energy (2022).
[5]
Huifang Kong, Jiapeng Yan, Hai Wang, and Lei Fan. 2020. Energy management strategy for electric vehicles based on deep Q-learning using Bayesian optimization. Neural Computing and Applications 32 (2020), 14431–14445.
[6]
Renzong Lian, Jiankun Peng, Yuankai Wu, Huachun Tan, and Hailong Zhang. 2020. Rule-interposing deep reinforcement learning based energy management strategy for power-split hybrid electric vehicle. Energy 197 (2020), 117297.
[7]
Teng Liu, Xiaosong Hu, Weihao Hu, and Yuan Zou. 2019. A heuristic planning reinforcement learning-based energy management for power-split plug-in hybrid electric vehicles. IEEE Transactions on Industrial Informatics 15, 12 (2019), 6436–6445.
[8]
Xiaodong Liu, Hongqiang Guo, Juan Du, and Xuan Zhao. 2022. A modified model-free-adaptive-control-based real-time energy management strategy for plug-in hybrid electric vehicle. Energy Science & Engineering 10, 10 (2022), 4007–4024.
[9]
Xiangnan Liu, Haijun Zhang, Keping Long, Arumugam Nallanathan, and Victor CM Leung. 2021. Deep Dyna-reinforcement learning based on random access control in LEO satellite IoT networks. IEEE Internet of Things Journal 9, 16 (2021), 14818–14828.
[10]
Clara Marina Martinez, Xiaosong Hu, Dongpu Cao, Efstathios Velenis, Bo Gao, and Matthias Wellers. 2016. Energy management in plug-in hybrid electric vehicles: Recent progress and a connected vehicles perspective. IEEE Transactions on Vehicular Technology 66, 6 (2016), 4534–4549.
[11]
Baolin Peng, Xiujun Li, Jianfeng Gao, Jingjing Liu, Kam-Fai Wong, and Shang-Yu Su. 2018. Deep dyna-q: Integrating planning for task-completion dialogue policy learning. arXiv preprint arXiv:1801.06176 (2018).
[12]
Fan Wang, Jie Gao, Mushu Li, and Lian Zhao. 2020. Autonomous PEV charging scheduling using Dyna-Q reinforcement learning. IEEE Transactions on Vehicular Technology 69, 11 (2020), 12609–12620.
[13]
Guanlin Wu, Wenqi Fang, Ji Wang, Jiang Cao, Weidong Bao, Yang Ping, Xiaomin Zhu, and Zheng Wang. 2021. Gaussian Process based Deep Dyna-Q approach for Dialogue Policy Learning. In Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021. 1786–1795.
[14]
Runnan Zou, Yuan Zou, Yanrui Dong, and Likang Fan. 2020. A self-adaptive energy management strategy for plug-in hybrid electric vehicle based on deep Q learning. In Journal of Physics: Conference Series, Vol. 1576. IOP Publishing, 012037.

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences
IC3-2023: Proceedings of the 2023 Fifteenth International Conference on Contemporary Computing
August 2023
783 pages
ISBN:9798400700224
DOI:10.1145/3607947
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 28 September 2023

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. Deep Reinforcement Learning
  2. Electric Vehicle
  3. Energy Management

Qualifiers

  • Research-article
  • Research
  • Refereed limited

Conference

IC3 2023

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • 0
    Total Citations
  • 35
    Total Downloads
  • Downloads (Last 12 months)14
  • Downloads (Last 6 weeks)2
Reflects downloads up to 15 Jan 2025

Other Metrics

Citations

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

HTML Format

View this article in HTML Format.

HTML Format

Media

Figures

Other

Tables

Share

Share

Share this Publication link

Share on social media