Abstract
This paper presents Chargym, a Python-based openai-gym compatible environment, that simulates the charging dynamics of a grid connected Electrical Vehicle (EV) charging station. Chargym transforms the classic EV charging problem into a Reinforcement Learning setup that can be used for benchmarking of various and off-the-shelf control and optimization algorithms enabling both single and multiple agent formulations. The incorporated charging station dynamics are presented with a brief explanation of the system parameters and function of the technical equipment. Moreover, we describe the structure of the used framework, highlighting the key features and data models that provide the necessary inputs for optimal control decisions. Finally, an experimental performance analysis is provided using two different state-of-the-art Reinforcement Learning (RL) algorithms validating the operation of the provided environment.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Ma, Z., Callaway, D.S., Hiskens, I.A.: Decentralized charging control of large populations of plug-in electric vehicles. IEEE Trans. Control Syst. Technol. 21(1), 67–78 (2011)
Han, S., Han, S.H., Sezaki, K.: Design of an optimal aggregator for vehicle-to-grid regulation service. In: 2010 Innovative Smart Grid Technologies (ISGT). IEEE (2010)
Korkas, C.D., Baldi, S., Michailidis, P., Kosmatopoulos, E.B.: A cognitive stochastic approximation approach to optimal charging schedule in electric vehicle stations. In: 2017 25th Mediterranean Conference on Control and Automation (MED), pp. 484–489. IEEE, July 2017
Korkas, C.D., Baldi, S., Yuan, S., Kosmatopoulos, E.B.: An adaptive learning-based approach for nearly optimal dynamic charging of electric vehicle fleets. IEEE Trans. Intell. Transp. Syst. 19(7), 2066–2075 (2017)
Qian, T., Shao, C., Wang, X., Shahidehpour, M.: Deep reinforcement learning for EV charging navigation by coordinating smart grid and intelligent transportation system. IEEE Trans. Smart Grid 11(2), 1714–1723 (2019)
Bhatti, A.R., et al.: Optimized sizing of photovoltaic grid-connected electric vehicle charging system using particle swarm optimization. Int. J. Energy Res. 43(1), 500–522 (2019)
Wan, Z., Li, H., He, H., Prokhorov, D.: Model-free real-time EV charging scheduling based on deep reinforcement learning. IEEE Trans. Smart Grid 10(5), 5246–5257 (2018)
Arif, S.M., Lie, T.T., Seet, B.C., Ayyadi, S., Jensen, K.: Review of electric vehicle technologies, charging methods, standards and optimization techniques. Electronics 10(16), 1910 (2021)
Zheng, Y., Song, Y., Hill, D.J., Meng, K.: Online distributed MPC-based optimal scheduling for EV charging stations in distribution systems. IEEE Trans. Ind. Inf. 15(2), 638–649 (2018)
Tang, W., Zhang, Y.J.: A model predictive control approach for low-complexity electric vehicle charging scheduling: optimality and scalability. IEEE Trans. Power Syst. 32(2), 1050–1063 (2016)
Zhang, M., Chen, J.: The energy management and optimized operation of electric vehicles based on microgrid. IEEE Trans. Power Deliv. 29(3), 1427–1435 (2014)
Bardi, M., Dolcetta, I.C.: Optimal Control and Viscosity Solutions of Hamilton-Jacobi-Bellman Equations, vol. 12. Birkhäuser, Boston (1997)
Rigas, E.S., Karapostolakis, S., Bassiliades, N., Ramchurn, S.D.: EVLibSim: a tool for the simulation of electric vehicles’ charging stations using the EVLib library. Simul. Model. Pract. Theory 87, 99–119 (2018)
Saxena, S.: Vehicle-to-grid Simulator (No. V2G-Sim; 005701MLTPL00). Lawrence Berkeley National Lab. (LBNL), Berkeley, CA (United States) (2013)
Lee, Z.J., Johansson, D., Low, S.H.: ACN-sim: an open-source simulator for data-driven electric vehicle charging research. In: 2019 IEEE International Conference on Communications, Control, and Computing Technologies for Smart Grids (SmartGridComm). IEEE (2019)
Díaz de Arcaya, A., et al.: Simulation platform for coordinated charging of electric vehicles (2015)
Strehler, M., Merting, S., Schwan, C.: Energy-efficient shortest routes for electric and hybrid vehicles. Transp. Res. Part B Methodol. 103, 111–135 (2017)
Mou, Y., et al.: Decentralized optimal demand-side management for PHEV charging in a smart grid. IEEE Trans. Smart Grid 6(2), 726–736 (2014)
Lillicrap, T.P., et al.: Continuous control with deep reinforcement learning (2015). arXiv preprint arXiv:1509.02971
Schulman, J., Wolski, F., Dhariwal, P., Radford, A., Klimov, O.: Proximal policy optimization algorithms (2017). arXiv preprint arXiv:1707.06347
Bae, S., Kwasinski, A.: Spatial and temporal model of electric vehicle charging demand. IEEE Trans. Smart Grid 3(1), 394–403 (2011)
Raffin, A., Hill, A., Gleave, A., Kanervisto, A., Ernestus, M., Dormann, N.: Stable-Baselines3: reliable reinforcement learning implementations. J. Mach. Learn. Res. (2021)
Acknowledgements
We acknowledge support of this work by the European Commission H2020-EU.2.1.5.2., Turning traditional reactive buildings into proactive ones, under contract 958284 (PRECEPT).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 IFIP International Federation for Information Processing
About this paper
Cite this paper
Karatzinis, G. et al. (2022). Chargym: An EV Charging Station Model for Controller Benchmarking. In: Maglogiannis, I., Iliadis, L., Macintyre, J., Cortez, P. (eds) Artificial Intelligence Applications and Innovations. AIAI 2022 IFIP WG 12.5 International Workshops. AIAI 2022. IFIP Advances in Information and Communication Technology, vol 652. Springer, Cham. https://doi.org/10.1007/978-3-031-08341-9_20
Download citation
DOI: https://doi.org/10.1007/978-3-031-08341-9_20
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-08340-2
Online ISBN: 978-3-031-08341-9
eBook Packages: Computer ScienceComputer Science (R0)
