Abstract
This paper presents intelligent access control for a charging station and a framework for dynamically and adaptively managing charging requests from randomly arriving electric vehicles (EVs), to increase the revenue of the station. First, charging service requests from random EV arrivals are described as an event-driven sequential decision process, and the decision-making relies on an event-extended state that is composed of the real-time electricity price, real-time charging station state, and EV arrival event. Second, a state aggregation method is introduced to reduce the state space by first aggregating the charging station state in the form of the remaining charging time and then further aggregating it via sort coding. Besides, mathematical calculations of the code value are provided, and their uniqueness and continuous integer characteristics are proved. Then, a corresponding Q-learning method is proposed to derive an optimal or suboptimal access control policy. The results of a case study demonstrate that the proposed learning optimisation method based on the event-extended state aggregation performs better than flat Q-learning. The space complexity and time complexity are significantly reduced, which substantially improves the learning efficiency and optimisation performance.
Similar content being viewed by others
References
Patil H and Kalkhambka V N, Grid integration of electric vehicles for economic benefits: A Review, Journal of Modern Power Systems and Clean Energy, 2021, 9(1): 13–26.
Tushar M H K, Zeineddine A W, and Assi C, Demand-side management by regulating charging and discharging of the EV, ESS, and utilizing renewable energy, IEEE Transactions on Industrial Informatics, 2018, 14(1): 117–126.
Nour M, Chaves-Ávila J P, Magdy G, et al., Review of positive and negative impacts of electric vehicles charging on electric power systems, Energies, 2020, 13(18): 4675.
Ding Z H, Lu Y, Lai K X, et al., Optimal coordinated operation scheduling for electric vehicle aggregator and charging stations in an integrated electricity-transportation system, International Journal of Electrical Power and Energy Systems, 2020, 121: 106040.
Zhou C B, Qi S Z, Zhang J H, et al., Potential co-benefit effect analysis of orderly charging and discharging of electric vehicles in China, Energy, 2021, 226: 120352.
Ye P, Li T Y, Sun F, et al., Research on ordered charging strategy of electric vehicle with double layer optimization method, IOP Conference Series Earth and Environmental Science, 2021, 645: 012019.
Chen G, Mao Z L, Li J Y, et al., Multi-objective optimal planning of electric vehicle charging stations considering carbon emission, Automation of Electric Power Systems, 2014, 38(17): 49–53, 136.
Izadkhast S, Aggregation of Plug-in Electric Vehicles in Power Systems for Primary Frequency Control, Universidad Pontificia Comillas, Spain, 2017.
Xu B W, Research on charging strategy optimization of electric vehicle based on AG, 5th International Conference on Vehicle, Mechanical and Electrical Engineering, Dalian, 2019.
Schinke A and Hirsch H, Impact of electric vehicle charging and photovoltaic generation on distribution system voltage volatility, 2019 IEEE Power Energy and Society General Meeting (PESGM), Atlanta, 2019.
Leterme W, Ruelens F, Claessens B, et al., A flexible stochastic optimization method for wind power balancing with PHEVs, IEEE Transactions on Smart Grid, 2014, 5(3): 1238–1245.
Ghotge R, Snow Y, Farahani S, et al., Optimized scheduling of ev charging in solar parking lots for local peak reduction under EV demand uncertainty, Energies, 2020, 13(5): 1275.
Diaz-Londono C, Colangelo L, Ruiz F, et al., Optimal strategy to exploit the flexibility of an electric vehicle charging station, Energies, 2019, 12(20): 3834.
Kasturi K, Nayak C K, and Nayak M R, Electric vehicles management enabling G2V and V2G in smart distribution system for maximizing profits using MOMVO, International Transactions on Electrical Energy Systems, 2019, 29(6): e12013.
Madina C, Zamora I, and Zabala E, Methodology for assessing electric vehicle charging infrastructure business models, Energy Policy, 2016, 89: 284–293.
Wenzel G, Negrete-Pincetic M, Olivares D E, et al., Real-time charging strategies for an electric vehicle aggregator to provide ancillary services, IEEE Transactions on Smart Grid, 2018, 9(5): 5141–5151.
Khemakhem S, Rekik M, and Krichen L, A flexible control strategy of plug-in electric vehicles operating in seven modes for smoothing load power curves in smart grid, Energy, 2017, 118(1): 197–208.
Vuelvas J, Ruiz F, and Gruosso G, A time-of-use pricing strategy for managing electric vehicle clusters, Sustainable Energy, Grids and Networks, 2021, 25: 100411.
Casini M, Vicino A, and Zanvettor G G, A receding horizon approach to peak power minimization for EV charging stations in the presence of uncertainty, International Journal of Electrical Power and Energy Systems, 2021, 126: 106567.
Liu C and Murphey Y L, Optimal power management based on Q-learning and neuro-dynamic programming for plug-in hybrid electric vehicles, IEEE Transactions on Neural Network and Learning Systems, 2020, 31(6): 1942–1954.
Lee Z J, Pang J Z F, and Low S H, Pricing EV charging service with demand charge, Electric Power Systems Research, 2020, 189: 106694.
Sutton R S and Barto A G, Reinforcement Learning: An Introduction, Second Edition, MIT Press, Cambridge, 2018.
Puterman M L, Markov Decision Processes: Discrete Stochastic Dynamic Programming, Wiley, New York, 1994.
Vuelvas J, Ruiz F, and Gruosso G, Limiting gaming opportunities on incentive-based demand response programs, Applied Energy, 2018, 225: 668–681.
Liu J J, Guo H Z, Xiong J Y, et al., Smart and resilient EV charging in SDN-enhanced vehicular edge computing networks, IEEE Journal on Selected Areas in Communications, 2020, 38(1): 217–228.
Chen L X, Huang X L, and Zhang H, Modeling the charging behaviors for electric vehicles based on ternary symmetric kernel density estimation, Energies, 2020, 13(7): 1551.
Author information
Authors and Affiliations
Corresponding authors
Additional information
This paper was supported by the National Natural Science Foundation of China under Grant Nos. 61871412, 61972439.
Rights and permissions
About this article
Cite this article
Tang, Z., Luo, Y., Fang, D. et al. Access Control Method for EV Charging Stations Based on State Aggregation and Q-Learning. J Syst Sci Complex 35, 2145–2165 (2022). https://doi.org/10.1007/s11424-022-1155-z
Received:
Revised:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11424-022-1155-z