Skip to main content
SpringerLink
Log in

On large action space in EV charging scheduling optimization

  • Research Paper
  • Published:
Science China Information Sciences Aims and scope Submit manuscript

Abstract

In order to reduce the air pollution by traditional fossil fuel and save the charging cost, much attention has been paid to the charging scheduling of electric vehicles (EVs) in the uncertain supplement of wind power and solar power. Owing to the randomness in the renewable power generation and the EV charging demand, simulation-based policy improvement (SBPI) has been well adopted for making charging decision. However, it is challenging to explore the large action space which grows exponentially with respect to the system scale. We consider this important problem in this paper and make the following contributions. First, we explore the structural property of the problem and develop an urgency index to rank the EVs. Second, we apply three methods to search in the action space. Third, we numerically demonstrate the performance of the urgency index and the search methods, and compare the SBPI methods with the CPLEX-based method. It shows that SBPI improves the base policies in all these cases.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Wang H W, Zhang X B, Ouyang M G. Energy and environmental life-cycle assessment of passenger car electrification based on Beijing driving patterns. Sci China Technol Sci, 2015, 58: 659–668

    Article  Google Scholar 

  2. Mohammadi K, Alavi O, Mostafaeipour A, et al. Assessing different parameters estimation methods of Weibull distribution to compute wind power density. Energy Convers Manage, 2016, 108: 322–335

    Article  Google Scholar 

  3. Murata A, Ohtake H, Oozeki T. Modeling of uncertainty of solar irradiance forecasts on numerical weather predictions with the estimation of multiple confidence intervals. Renew Energy, 2018, 117: 193–201

    Article  Google Scholar 

  4. Amjad M, Ahmad A, Rehmani M H, et al. A review of EVs charging: from the perspective of energy optimization, optimization approaches, and charging techniques. Transpation Res Part D-Transp Environ, 2018, 62: 386–417

    Article  Google Scholar 

  5. Nimalsiri N I, Mediwaththe C P, Ratnam E L, et al. A survey of algorithms for distributed charging control of electric vehicles in smart grid. IEEE Trans Intell Transp Syst, 2020, 21: 4497–4515

    Article  Google Scholar 

  6. Jia Q S, Shen J X, Xu Z B, et al. Simulation-based policy improvement for energy management in commercial office buildings. IEEE Trans Smart Grid, 2012, 3: 2211–2223

    Article  Google Scholar 

  7. Xu Y, Pan F, Tong L. Dynamic scheduling for charging electric vehicles: a priority rule. IEEE Trans Automat Contr, 2016, 61: 4094–4099

    Article  MathSciNet  MATH  Google Scholar 

  8. Hu J Q, Fu M C, Marcus S I. A model reference adaptive search method for global optimization. Operations Res, 2007, 55: 549–568

    Article  MathSciNet  MATH  Google Scholar 

  9. Hu J Q, Fu M C, Marcus S I. A model reference adaptive search method for stochastic global optimization. Commun Inf Syst, 2008, 8: 245–276

    Article  MathSciNet  MATH  Google Scholar 

  10. Hu J Q, Hu P, Chang H S. A stochastic approximation framework for a class of randomized optimization algorithms. IEEE Trans Automat Contr, 2012, 57: 165–178

    Article  MathSciNet  MATH  Google Scholar 

  11. Hong L J, Nelson B L. Discrete optimization via simulation using COMPASS. Operations Res, 2006, 54: 115–129

    Article  MATH  Google Scholar 

  12. Yan Q, Zhang B, Kezunovic M. Optimized operational cost reduction for an EV charging station integrated with battery energy storage and PV generation. IEEE Trans Smart Grid, 2019, 10: 2096–2106

    Article  Google Scholar 

  13. Zhong Q W, Buckley S, Vassallo A, et al. Energy cost minimization through optimization of EV, home and workplace battery storage. Sci China Technol Sci, 2018, 61: 761–773

    Article  Google Scholar 

  14. Zhang Y M, Wei Z, Li H, et al. Optimal charging scheduling for catenary-free trams in public transportation systems. IEEE Trans Smart Grid, 2019, 10: 227–237

    Article  Google Scholar 

  15. Eldeeb H H, Faddel S, Mohammed O A. Multi-objective optimization technique for the operation of grid tied PV powered EV charging station. Electric Power Syst Res, 2018, 164: 201–211

    Article  Google Scholar 

  16. Zhang Y M, Cai L. Dynamic charging scheduling for EV parking lots with photovoltaic power system. IEEE Access, 2018, 6: 56995–57005

    Article  Google Scholar 

  17. Jia Q S, Wu J. On distributed event-based optimization for shared economy in cyber-physical energy systems. Sci China Inf Sci, 2018, 61: 110203

    Article  Google Scholar 

  18. Xie S, Zhong W, Xie K, et al. Fair energy scheduling for vehicle-to-grid networks using adaptive dynamic programming. IEEE Trans Neural Netw Learn Syst, 2016, 27: 1697–1707

    Article  MathSciNet  Google Scholar 

  19. Wu Y, Ravey A, Chrenko D, et al. Demand side energy management of EV charging stations by approximate dynamic programming. Energy Convers Manage, 2019, 196: 878–890

    Article  Google Scholar 

  20. Zhang Y M, You P C, Cai L. Optimal charging scheduling by pricing for EV charging station with dual charging modes. IEEE Trans Intell Transp Syst, 2019, 20: 3386–3396

    Article  Google Scholar 

  21. Huang Q, Jia Q S, Guan X. A multi-timescale and bilevel coordination approach for matching uncertain wind supply with EV charging demand. IEEE Trans Automat Sci Eng, 2017, 14: 694–704

    Article  Google Scholar 

  22. Puterman M, Puterman M L. Markov Decision Processes: Discrete Stochastic Dynamic Programming. Hoboken: John Wiley & Sons, 2014

    MATH  Google Scholar 

  23. Hastings W K. Monte Carlo sampling methods using Markov chains and their applications. Biometrika, 1970, 57: 97–109

    Article  MathSciNet  MATH  Google Scholar 

  24. Leterme W, Ruelens F, Claessens B, et al. A flexible stochastic optimization method for wind power balancing with PHEVs. IEEE Trans Smart Grid, 2014, 5: 1238–1245

    Article  Google Scholar 

  25. Wang Z, Jochem P, Fichtner W. A scenario-based stochastic optimization model for charging scheduling of electric vehicles under uncertainties of vehicle availability and charging demand. J Cleaner Production, 2020, 254: 119886

    Article  Google Scholar 

  26. Lopez K L, Gagne C, Gardner M A. Demand-side management using deep learning for smart charging of electric vehicles. IEEE Trans Smart Grid, 2019, 10: 2683–2691

    Article  Google Scholar 

  27. Wan Z, Li H, He H, et al. Model-free real-time EV charging scheduling based on deep reinforcement learning. IEEE Trans Smart Grid, 2019, 10: 5246–5257

    Article  Google Scholar 

  28. Qian T, Shao C, Wang X, et al. Deep reinforcement learning for EV charging navigation by coordinating smart grid and intelligent transportation system. IEEE Trans Smart Grid, 2020, 11: 1714–1723

    Article  Google Scholar 

  29. Yang Y, Jia Q S, Deconinck G, et al. Distributed coordination of EV charging with renewable energy in a microgrid of buildings. IEEE Trans Smart Grid, 2018, 9: 6253–6264

    Article  Google Scholar 

  30. Chang H S, Hu J, Fu M C, et al. Simulation-based Algorithms for Markov Decision Processes. London: Springer, 2013

    Book  MATH  Google Scholar 

  31. Sedighizadeh M, Mohammadpour A, Alavi S M M. A daytime optimal stochastic energy management for EV commercial parking lots by using approximate dynamic programming and hybrid big bang big crunch algorithm. Sustain Cities Soc, 2019, 45: 486–498

    Article  Google Scholar 

  32. Tushar M H K, Assi C, Maier M, et al. Smart microgrids: optimal joint scheduling for electric vehicles and home appliances. IEEE Trans Smart Grid, 2014, 5: 239–250

    Article  Google Scholar 

  33. Blanco M I. The economics of wind energy. Renew Sustain Energy Rev, 2009, 13: 1372–1382

    Article  Google Scholar 

  34. Branker K, Pathak M J M, Pearce J M. A review of solar photovoltaic levelized cost of electricity. Renew Sustain Energy Rev, 2011, 15: 4470–4482

    Article  Google Scholar 

  35. Grogg K. Harvesting the wind: the physics of wind turbines. Physics and Astronomy Comps Papers. Northfield: Carleton College, 2005

    Google Scholar 

  36. Ishaque K, Salam Z, Syafaruddin Z. A comprehensive MATLAB simulink PV system simulator with partial shading capability based on two-diode model. Sol Energy, 2011, 85: 2217–2227

    Article  Google Scholar 

  37. Shahidinejad S, Bibeau E, Filizadeh S. Statistical development of a duty cycle for plug-in vehicles in a north american urban setting using fleet information. IEEE Trans Veh Technol, 2010, 59: 3710–3719

    Article  Google Scholar 

  38. Bertsekas D P, Castanon D A. Rollout algorithms for stochastic scheduling problems. J Heuristics, 1999, 5: 89–108

    Article  MATH  Google Scholar 

  39. Huang Q, Jia Q S, Qiu Z, et al. Matching EV charging load with uncertain wind: a simulation-based policy improvement approach. IEEE Trans Smart Grid, 2015, 6: 1425–1433

    Article  Google Scholar 

  40. Lee T K, Bareket Z, Gordon T, et al. Stochastic modeling for studies of real-world PHEV usage: driving schedule and daily temporal distributions. IEEE Trans Veh Technol, 2012, 61: 1493–1502

    Article  Google Scholar 

Download references

Acknowledgements

This work was supported in part by National Key Research and Development Program of China (Grant No. 2016YFB0901900), National Natural Science Foundation of China (Grant No. 61673229), and the 111 International Collaboration Project of China (Grant No. BP2018006).

Author information

Authors and Affiliations

  1. Center for Intelligent and Networked Systems, Beijing National Research Center for Information Science and Technology, Department of Automation, Tsinghua University, Beijing, 100084, China

    Zhaoyu Jiang, Qing-Shan Jia & Xiaohong Guan

  2. MOE KLINNS Lab, Xi’an Jiaotong University, Xi’an, 710049, China

    Xiaohong Guan

Authors
  1. Zhaoyu Jiang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Qing-Shan Jia
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Xiaohong Guan
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Qing-Shan Jia.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Jiang, Z., Jia, QS. & Guan, X. On large action space in EV charging scheduling optimization. Sci. China Inf. Sci. 65, 122201 (2022). https://doi.org/10.1007/s11432-020-3106-7

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s11432-020-3106-7

Keywords

Search

Navigation