Muddsair Sharif
ABSTRACT
The expansion of charging infrastructure and the optimal utilization of existing infrastructure are key influencing factors for the future growth of electric mobility. The main objective of this paper is to present a novel methodology which identifies the necessary stakeholders, processes their contextual information and meets their optimality criteria using a constraint satisfaction strategy. A deep reinforcement learning algorithm is used for optimally distributing the electric vehicle charging resources in a smart-mobility ecosystem. The algorithm performs context-aware, constrained-optimization such that the on-demand requests of each stakeholder, e.g., vehicle owner as end-user, grid-operator, fleet-operator, charging-station service operator, is fulfilled. In the proposed methodology, the system learns from the surrounding environment until the optimal charging resource allocation strategy within the limitations of the system constraints is reached. We look at the concept of optimality from the perspective of multiple stakeholders who participate in the smart-mobility ecosystem. A simple use case is presented in detail. Finally, we discuss the potential to develop this concept further to enable more complex digital interactions between the actors of a smart-mobility eco-system.
- VDA. Electric Mobility: Electric Mobility in Germany, 2020.Google Scholar
- VDA. Electric Mobility: Charging Infrastructure, 2020.Google Scholar
- Philipp Danner, Wolfgang Duschl, Dominik Danner, Ammar Alyousef, and Hermann de Meer. Flexibility reward scheme for grid-friendly electric vehicle charging in the distribution power grid. In Proceedings of the Ninth International Conference on Future Energy Systems, eEnergy '18, page 564--569, New York, NY, USA, 2018. Association for Computing Machinery.Google ScholarDigital Library
- Hussein Dia. Rethinking Urban Mobility: Unlocking the Benefits of Vehicle Electrification. In Peter Newton, Deo Prasad, Alistair Sproul, and Stephen White, editors, Decarbonising the Built Environment, volume 2416, pages 83--98. Springer Singapore, Singapore, 2019.Google ScholarCross Ref
- Ngoc Duy Nguyen, Thanh Thi Nguyen, Hai Nguyen, and Saeid Nahavandi. Review, analyze, and design a comprehensive deep reinforcement learning framework, 2020.Google Scholar
- VincentFrançois-Lavet, PeterHenderson, RiashatIslam, MarcG.Belle-mare, and Joelle Pineau. An introduction to deep reinforcement learning. CoRR, abs/1811.12560, 2018.Google Scholar
- Joshua Achiam, David Held, Aviv Tamar, and Pieter Abbeel. Constrained policy optimization, 2017.Google Scholar
- JaneLin, WeiZhou, andOuriWolfson.Electricvehicleroutingp roblem. Transportation Research Procedia, 12(Supplement C):508--521, 2016.Google Scholar
- Tao Chen, Bowen Zhang, Hajir Pourbabak, Abdollah Kavousi-Fard, and Wencong Su. Optimal routing and charging of an electric vehicle fleet for high-efficiency dynamic transit systems. IEEE Transactions on Smart Grid, 9(4):3563--3572, 2016.Google ScholarCross Ref
- Sandford Bessler and Jesper Grønbæk. Routing EV users towards an optimal charging plan. World Electric Vehicle Journal, 5(3):688--695, 2012.Google ScholarCross Ref
- Yunhao Tang, Shipra Agrawal, and Yuri Faenza. Reinforcement learning for integer programming: Learning to cut. CoRR, abs/1906.04859, 2019.Google Scholar
- Claudio Biancalana, Andrea Flamini, Fabio Gasparetti, Alessandro Micarelli, Samuele Millevolte, and Giuseppe Sansonetti. Enhancing traditional local search recommendations with context-awareness. In Joseph A. Konstan, Ricardo Conejo, José L. Marzo, and Nuria Oliver, editors, User Modeling, Adaption and Personalization, pages 335--340, Berlin, Heidelberg, 2011. Springer Berlin Heidelberg.Google ScholarCross Ref
- Ammar Alyousef, Friederich Kupzog, Dominik Danner, and Hermann de Meer. Enhancing power quality in electrical distribution systems using a smart charging architecture. New York, NY, USA, 2018.Google Scholar
- K. Li, T. Zhang, and R. Wang. Deep reinforcement learning for multi-objective optimization. IEEE Transactions on Cybernetics, pages 1--12, 2020.Google Scholar
- Thanh Thi Nguyen. A multi-objective deep reinforcement learning framework. CoRR, abs/1803.02965, 2018.Google Scholar
- Lasse Espeholt, Hubert Soyer, Remi Munos, Karen Simonyan, Vlad Mnih, Tom Ward, Yotam Doron, Vlad Firoiu, Tim Harley, Iain Dunning, Shane Legg, and Koray Kavukcuoglu. IMPALA: Scalable distributed deep-RL with importance weighted actor-learner architectures. In Jennifer Dy and Andreas Krause, editors, Proceedings of the 35th International Conference on Machine Learning, volume 80 of Proceedings of Machine Learning Research, pages 1407--1416, Stockholmsmässan, Stockholm Sweden, 10--15 Jul 2018. PMLR.Google Scholar
- Zihan Jiang, Wanling Gao, Lei Wang, Xingwang Xiong, Yuchen Zhang, Xu Wen, Chunjie Luo, Hainan Ye, Xiaoyi Lu, Yunquan Zhang, Shengzhong Feng, Kenli Li, Weijia Xu, and Jianfeng Zhan. Hpc ai500: A benchmark suite for hpc ai systems. In Chen Zheng and Jianfeng Zhan, editors, Benchmarking, Measuring, and Optimizing, pages 10--22, Cham, 2019. Springer International Publishing.Google Scholar
- K. S. Amogh Vardhan, M. Jakaraddi, D. S. G., J. Shetty, A. Chala, and D. Camper. Design and development of iot plugin for hpcc systems. In 2019 IEEE 4th International Conference on Big Data Analytics (ICBDA), pages 158--162, 2019.Google ScholarCross Ref
- Muddsair Sharif, Siegfried Mercelis, Wim Van Den Bergh, and Peter Hellinckx. Towards real-time smart road construction: Efficient process management through the implementation of internet of things. In Proceedings of the International Conference on Big Data and Internet of Thing, BDIOT2017, page 174--180, New York, NY, USA, 2017. Association for Computing Machinery.Google ScholarDigital Library
Index Terms
- Context-Aware Optimal Charging Distribution using Deep Reinforcement Learning
Recommendations
Electric Vehicle Charging Navigation Strategy Based on Data Driven and Deep Reinforcement Learning
CCEAI '21: Proceedings of the 5th International Conference on Control Engineering and Artificial IntelligenceWith the popularization and popularization of electric vehicles, the interaction between electric vehicles and power grid and transportation networks is getting deeper and deeper. Considering the multi-data fusion characteristics of dynamic driving ...
A Novel Model based Energy Management Strategy for Plug-in Hybrid Electric Vehicles using Deep Reinforcement Learning
IC3-2023: Proceedings of the 2023 Fifteenth International Conference on Contemporary ComputingOver the last few years, Hybrid Electric Vehicles (HEVs) have become increasingly popular due to their potential to simplify fuel consumption and greenhouse gas emissions. The energy management of HEVs is a critical task that involves controlling the ...
Bi-level optimization of charging scheduling of a battery swap station based on deep reinforcement learning
AbstractWith the rapid increase of in the number of electric vehicle (EV), battery swapping is becoming a promising idea because of its short service waiting time. However, in the face of the uncertainty of the power grid and EV behavior, it ...
Highlights- A bi-level optimization model for large-scale battery charging scheduling in BSS.
Comments