Skip to main content

Advertisement

SpringerLink
Log in

Access Control Method for EV Charging Stations Based on State Aggregation and Q-Learning

  • Published:
Journal of Systems Science and Complexity Aims and scope Submit manuscript

Abstract

This paper presents intelligent access control for a charging station and a framework for dynamically and adaptively managing charging requests from randomly arriving electric vehicles (EVs), to increase the revenue of the station. First, charging service requests from random EV arrivals are described as an event-driven sequential decision process, and the decision-making relies on an event-extended state that is composed of the real-time electricity price, real-time charging station state, and EV arrival event. Second, a state aggregation method is introduced to reduce the state space by first aggregating the charging station state in the form of the remaining charging time and then further aggregating it via sort coding. Besides, mathematical calculations of the code value are provided, and their uniqueness and continuous integer characteristics are proved. Then, a corresponding Q-learning method is proposed to derive an optimal or suboptimal access control policy. The results of a case study demonstrate that the proposed learning optimisation method based on the event-extended state aggregation performs better than flat Q-learning. The space complexity and time complexity are significantly reduced, which substantially improves the learning efficiency and optimisation performance.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Patil H and Kalkhambka V N, Grid integration of electric vehicles for economic benefits: A Review, Journal of Modern Power Systems and Clean Energy, 2021, 9(1): 13–26.

    Article  Google Scholar 

  2. Tushar M H K, Zeineddine A W, and Assi C, Demand-side management by regulating charging and discharging of the EV, ESS, and utilizing renewable energy, IEEE Transactions on Industrial Informatics, 2018, 14(1): 117–126.

    Article  Google Scholar 

  3. Nour M, Chaves-Ávila J P, Magdy G, et al., Review of positive and negative impacts of electric vehicles charging on electric power systems, Energies, 2020, 13(18): 4675.

    Article  Google Scholar 

  4. Ding Z H, Lu Y, Lai K X, et al., Optimal coordinated operation scheduling for electric vehicle aggregator and charging stations in an integrated electricity-transportation system, International Journal of Electrical Power and Energy Systems, 2020, 121: 106040.

    Article  Google Scholar 

  5. Zhou C B, Qi S Z, Zhang J H, et al., Potential co-benefit effect analysis of orderly charging and discharging of electric vehicles in China, Energy, 2021, 226: 120352.

    Article  Google Scholar 

  6. Ye P, Li T Y, Sun F, et al., Research on ordered charging strategy of electric vehicle with double layer optimization method, IOP Conference Series Earth and Environmental Science, 2021, 645: 012019.

    Article  Google Scholar 

  7. Chen G, Mao Z L, Li J Y, et al., Multi-objective optimal planning of electric vehicle charging stations considering carbon emission, Automation of Electric Power Systems, 2014, 38(17): 49–53, 136.

    Google Scholar 

  8. Izadkhast S, Aggregation of Plug-in Electric Vehicles in Power Systems for Primary Frequency Control, Universidad Pontificia Comillas, Spain, 2017.

    Google Scholar 

  9. Xu B W, Research on charging strategy optimization of electric vehicle based on AG, 5th International Conference on Vehicle, Mechanical and Electrical Engineering, Dalian, 2019.

  10. Schinke A and Hirsch H, Impact of electric vehicle charging and photovoltaic generation on distribution system voltage volatility, 2019 IEEE Power Energy and Society General Meeting (PESGM), Atlanta, 2019.

  11. Leterme W, Ruelens F, Claessens B, et al., A flexible stochastic optimization method for wind power balancing with PHEVs, IEEE Transactions on Smart Grid, 2014, 5(3): 1238–1245.

    Article  Google Scholar 

  12. Ghotge R, Snow Y, Farahani S, et al., Optimized scheduling of ev charging in solar parking lots for local peak reduction under EV demand uncertainty, Energies, 2020, 13(5): 1275.

    Article  Google Scholar 

  13. Diaz-Londono C, Colangelo L, Ruiz F, et al., Optimal strategy to exploit the flexibility of an electric vehicle charging station, Energies, 2019, 12(20): 3834.

    Article  Google Scholar 

  14. Kasturi K, Nayak C K, and Nayak M R, Electric vehicles management enabling G2V and V2G in smart distribution system for maximizing profits using MOMVO, International Transactions on Electrical Energy Systems, 2019, 29(6): e12013.

    Article  Google Scholar 

  15. Madina C, Zamora I, and Zabala E, Methodology for assessing electric vehicle charging infrastructure business models, Energy Policy, 2016, 89: 284–293.

    Article  Google Scholar 

  16. Wenzel G, Negrete-Pincetic M, Olivares D E, et al., Real-time charging strategies for an electric vehicle aggregator to provide ancillary services, IEEE Transactions on Smart Grid, 2018, 9(5): 5141–5151.

    Article  Google Scholar 

  17. Khemakhem S, Rekik M, and Krichen L, A flexible control strategy of plug-in electric vehicles operating in seven modes for smoothing load power curves in smart grid, Energy, 2017, 118(1): 197–208.

    Article  Google Scholar 

  18. Vuelvas J, Ruiz F, and Gruosso G, A time-of-use pricing strategy for managing electric vehicle clusters, Sustainable Energy, Grids and Networks, 2021, 25: 100411.

    Article  Google Scholar 

  19. Casini M, Vicino A, and Zanvettor G G, A receding horizon approach to peak power minimization for EV charging stations in the presence of uncertainty, International Journal of Electrical Power and Energy Systems, 2021, 126: 106567.

    Article  Google Scholar 

  20. Liu C and Murphey Y L, Optimal power management based on Q-learning and neuro-dynamic programming for plug-in hybrid electric vehicles, IEEE Transactions on Neural Network and Learning Systems, 2020, 31(6): 1942–1954.

    Article  Google Scholar 

  21. Lee Z J, Pang J Z F, and Low S H, Pricing EV charging service with demand charge, Electric Power Systems Research, 2020, 189: 106694.

    Article  Google Scholar 

  22. Sutton R S and Barto A G, Reinforcement Learning: An Introduction, Second Edition, MIT Press, Cambridge, 2018.

    MATH  Google Scholar 

  23. Puterman M L, Markov Decision Processes: Discrete Stochastic Dynamic Programming, Wiley, New York, 1994.

    Book  MATH  Google Scholar 

  24. Vuelvas J, Ruiz F, and Gruosso G, Limiting gaming opportunities on incentive-based demand response programs, Applied Energy, 2018, 225: 668–681.

    Article  Google Scholar 

  25. Liu J J, Guo H Z, Xiong J Y, et al., Smart and resilient EV charging in SDN-enhanced vehicular edge computing networks, IEEE Journal on Selected Areas in Communications, 2020, 38(1): 217–228.

    Article  Google Scholar 

  26. Chen L X, Huang X L, and Zhang H, Modeling the charging behaviors for electric vehicles based on ternary symmetric kernel density estimation, Energies, 2020, 13(7): 1551.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. School of Computer and Information, Anhui Normal University, Wuhu, 241002, China

    Ziyu Tang, Yonglong Luo & Chuanxin Zhao

  2. Electrical Engineering and Automation, Hefei University of Technology, Hefei, 230009, China

    Daohong Fang

Authors
  1. Ziyu Tang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yonglong Luo
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Daohong Fang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Chuanxin Zhao
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Ziyu Tang, Yonglong Luo, Daohong Fang or Chuanxin Zhao.

Additional information

This paper was supported by the National Natural Science Foundation of China under Grant Nos. 61871412, 61972439.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Tang, Z., Luo, Y., Fang, D. et al. Access Control Method for EV Charging Stations Based on State Aggregation and Q-Learning. J Syst Sci Complex 35, 2145–2165 (2022). https://doi.org/10.1007/s11424-022-1155-z

Download citation

  • Received:

  • Revised:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11424-022-1155-z

Keywords

Search

Navigation