Skip to main content

Advertisement

Springer Nature Link
Log in

F-DQN: an optimized DQN for decision-making of generator start-up sequence after blackout

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

The decision-making of generator start-up sequence plays a pivotal role in the power system restoration process following the blackout. In this paper, an optimized deep Q-learning network (DQN) algorithm is proposed to address this challenge. The generator start-up process is modeled as a Markov Decision Process (MDP) based on its characteristics. The DQN is tasked with deciding both the generator start-up sequence and the corresponding restoration path. To address the limitations of DQN, such as low exploration efficiency and slow convergence, the study incorporates the Artificial Potential Field (APF) algorithm to refine the reward function of it. This integration results in the development of the F-DQN (APF-DQN) algorithm, which enhances training efficiency. The effectiveness of this proposed method is demonstrated through the IEEE 39-bus test system. The results reveal that the DQN algorithm is capable of efficiently solving the model of the generator start-up sequence after the blackout. Moreover, the F-DQN algorithm exhibits superior learning efficiency, faster convergence, and higher-quality optimal solutions compared to the DQN. This paper also discusses the applicability of this method under partial blackouts. When compared to other decision-making algorithms, the proposed method offers a restoration scheme that is both time-efficient and results in increased electricity generation.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Algorithm 1
Fig. 8
Algorithm 2
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

Data availability

My manuscript has no associated data.

References

  1. Khaledi A, Saifoddin A (2023) Three-stage resilience-oriented active distributionsystems operation after natural disasters. Energy 282:128360

    Article  Google Scholar 

  2. Li S, Lin Z, Zhang Y, Gu X, Wang H (2023) Optimization method of skeleton network partitioning scheme considering resilience active improvement in power system restoration after typhoon passes through. Int J Electr Power Energy Syst 148:109001

    Article  Google Scholar 

  3. Liu X, Wang H, Sun Q, Guo T (2022) Research on fault scenario prediction andresilience enhancement strategy of active distribution network under ice disaster. Int JElectr Power Energy Syst 135:107478

    Article  Google Scholar 

  4. Zhang Z, Zuo K, Deng R, Teng F, Sun M (2023) Cybersecurity analysis of data-driven power system stability assessment. IEEE Internet Things J 10(17):15723–15735

    Article  Google Scholar 

  5. Liang K, Wang H, Pozo D, Terzija V (2024) Power system restoration with large renewable penetration: state-of-the-art and future trends. Int J Electr Power EnergySyst 155:109494

    Article  Google Scholar 

  6. Wang J, Pinson P, Chatzivasileiadis S, Panteli M, Strbac G, Terzija V (2023) On machine learning-based techniques for future sustainable and resilient energy systems. IEEE Trans Sustain Energy 14(2):1230–1243

    Article  Google Scholar 

  7. Kumar S, Pandey A, Goswami P, Pentayya P, Kazi F (2022) Analysis of Mumbai grid failure restoration on Oct 12, 2020: challenges and lessons Learnt. IEEE Trans Power Syst 37(6):4555–4567

    Article  Google Scholar 

  8. Guanglun Z, Haiwang Z, Zhenfei T, Tong C, Qing X, Chongqing K (2022) Texas electric power crisis of 2021 warns of a new blackout mechanism. CSEE J Power Energy Syst 16(9):1–9

    Google Scholar 

  9. Du Y, Tu H, Lu X, Wang J, Lukic S (2022) Black-start and service restoration in resilient distribution systems with dynamic microgrids. IEEE J Emerg Sel Top Power Electron 10(4):3975–3986

    Article  Google Scholar 

  10. Liu Y, Fan R, Terzija V (2016) Power system restoration: a literature review from 2006 to 2016. J Mod Power Syst Clean Energy 4(3):332–341

    Article  Google Scholar 

  11. Ganganath N, Wang JV, Xu X, Cheng C-T, Tse CK (2018) Agglomerative clustering-based network partitioning for parallel Power System Restoration. IEEE Trans Industr Inf 14(8):3325–3333

    Article  Google Scholar 

  12. Xiang Z, Meng L, Chengxiang L, Tianyu L, Ye W, Cunzhi T, Xiang M, Xiaocong Z, Chen Z (2022) Evaluation of black-start schemes based on the evaluation indicator system. Int Trans Electr Energy Syst 2022:1–11

    Google Scholar 

  13. Zhang H, Mengke L, Xiaobo K, Shaoqi Y, Jianzhe Z, Ye W, Lin C, Xiaoteng L (2021) Evaluation model of black-start schemes based on optimal combination weights and improved VIKOR method. Int J Electr Power Energy Syst 129:106762

  14. Granberg D, Pinney D, Eldali F (2022) An optimal algorithmic approach to efficiently automate fault isolation and service restoration on an arbitrary distribution feeder system. IEEE Trans Power Delivery 37(4):3006–3015

    Article  Google Scholar 

  15. Zhang C, Lin Z, Wen F, Ledwich G, Xue Y (2014) Two-stage power network reconfiguration strategy considering node importance and restored generation capacity. IET Gener Transm Distrib 8(1):91–103

    Article  Google Scholar 

  16. Gu X, Bai Y, Li S, Liu K, Liu Y, Wang H (2023) An optimisation method of whole-process restoration decision‐making of power systems considering disturbance‐resisting ability of the restored network. IET Gener Transm Distrib 17(7):1638–1651

    Article  Google Scholar 

  17. Wang Z, Zekai W, Tao D, Chenggang M, Yuhan H, Miao Y, Yueyang Y, Zhiming C, Yin L, Meng L (2024) An ADMM-based power system partitioned black-start and parallel restoration method considering high-penetrated renewable energy. Int J Electr Power Energy Syst 155:109532

    Article  Google Scholar 

  18. Du Y, Wu D (2022) Deep reinforcement learning from demonstrations to assist service restoration in islanded microgrids. IEEE Trans Sustain Energy 13(2):1062–1072

    Article  Google Scholar 

  19. Leng Y-J, Huang Y-H (2022) Power system black-start decision making based on back-propagation neural network and genetic algorithm. J Electr Eng Technol 17(4):2123–2134

    Article  Google Scholar 

  20. Li C, Ye Y, Huang S, Xu Y, Wang B, Gao CW (2022) Online decision-making of parallel restoration strategy for power systems based on susceptible-infected-recovered model. Int Trans Electr Energy Syst 2022:1–14

    Google Scholar 

  21. Ahmadipour M, Murtadha Othman M, Salam Z, Alrifaey M, Mohammed Ridha H, Veerasamy V (2023) Optimal load shedding scheme using grasshopper optimization algorithm for islanded power system with distributed energy resources. Ain Shams Eng J 14(1):101835

    Article  Google Scholar 

  22. Alobaidi AH, Fazlhashemi SS, Khodayar M, Wang J, Khodayar ME (2023) Distribution service restoration with renewable energy sources: a review. IEEE Trans Sustain Energy 14(2):1151–1168

    Article  Google Scholar 

  23. Shi J, Wang B, Yuan R, Wang Z, Chen C, Watada J (2023) Rolling horizon wind-thermal unit commitment optimization based on deep reinforcement learning. Appl Intell 53(16):19591–19609

    Article  Google Scholar 

  24. Kumar V, Goswami S, Smith D, Karniadakis GE (2023) Real-time prediction of gas flow dynamics in diesel engines using a deep neural operator framework. Appl Intell 54(1):14–34

    Article  Google Scholar 

  25. Xiang Y, Wang T, Wang Z (2023) Risk prediction based preventive islanding scheme for power system under typhoon involved with rainstorm events. IEEE Trans Power Syst 38(5):4177–4190

    Article  Google Scholar 

  26. Lin B, Wang H, Zhang Y, Wen B (2022) Real-time power system generator trippingcontrol based on deep reinforcement learning. Int J Electr Power Energy Syst 141:108127

    Article  Google Scholar 

  27. Ma H, Lei X, Li Z, Yu S, Liu B, Dong X (2023) Deep-learning based power system events detection technology using spatio-temporal and frequency information. IEEE J Emerg Sel Top Circuits Syst 13(2):545–556

    Article  Google Scholar 

  28. Li C, Xu W, Huang S, Yang L (2020) A Monte Carlo tree searchbased method foronline decision making of generator startup sequence considering hot start. Int JElectr Power Energy Syst 121:106070

    Article  Google Scholar 

  29. Jiang W, Liu Y, Fang G, Ding Z (2023) Research on short-term optimal scheduling of hydro-wind-solar multi-energy power system based on deep reinforcement learning. J Clean Prod 385:135704

    Article  Google Scholar 

  30. Liang Y, Ding Z, Zhao T, Lee W-J (2023) Real-time operation management for battery swapping-charging system via multi-agent deep reinforcement learning. IEEE Trans Smart Grid 14(1):559–571

    Article  Google Scholar 

  31. Hassani H, Razavi-Far R, Saif M (2022) Real-time out-of-step prediction control to prevent emerging blackouts in power systems: a reinforcement learning approach. Appl Energy 314:118861

    Article  Google Scholar 

  32. Lee J, Yeom H, Lee S-H, Ha J (2024) Channel correlation in multi-user covert communication: friend or foe? IEEE Trans Inf Forensics Secur 19:1469–1482

    Article  Google Scholar 

  33. Ismail H, Serhani MA, Hussein NM, Elhadef M (2023) RL-ECGNet: resource-aware multi-class detection of arrhythmia through reinforcement learning. Appl Intell 53(24):30927–30939

    Article  Google Scholar 

Download references

Acknowledgements

This work was supported by the Natural Science Foundation of Guangxi Province under Grant 2020GXNSFBA297069.

Author information

Authors and Affiliations

  1. Guangxi Key Laboratory of Power System Optimization and Energy Technology, Nanning, China

    Changcheng Li & Zirui Wu

Authors
  1. Changcheng Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Zirui Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

Changcheng Li (First Author): Conceptualization, Methodology, Software, Investigation, Formal Analysis, Writing - Original Draft.

Zirui Wu (Corresponding Author): Conceptualization, Funding Acquisition, Resources, Supervision, Writing - Review & Editing.

Corresponding author

Correspondence to Zirui Wu.

Ethics declarations

Ethical and informed consent for data used

This study did not involve ethical approval.

Competing interests

The authors declared that they have no conflicts of interest to this work.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Li, C., Wu, Z. F-DQN: an optimized DQN for decision-making of generator start-up sequence after blackout. Appl Intell 54, 5521–5535 (2024). https://doi.org/10.1007/s10489-024-05392-3

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-024-05392-3

Keywords